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ABSTRACT: 

The  kinematic  aspect  of  surveillance-evasion  is  studied  with  a 
deterministic  differential  game  model.   The  model  considers  a  Pursuer 
with  limitations  on  both  speed  and  maneuverability  (turning  radius) 
and  an  Evader  with  only  a  speed  limitation.   Conditions  are  developed 
for  the  Pursuer  to  be  able  to  maintain  contact  indefinitely.   The 
results  of  this  research  modify  previously  published  results  on  this 
problem.   Shortcomings  of  previous  work  are  discussed  including  the 
fact  that  the  surveillance-evasion  problem  has  not  been  solved  for  an 
arbitrary  detection  region.   Related  parts  of  the  solution  to  Isaacs' 
homicidal  chauffeur  game  and  its  one-sided  counterpart  are  developed 
as  background  material.   Some  known  allocation  of  effort  in  search 
theory  results  are  derived  by  the  Pontryagin  maximum  principle. 
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I .   Introduction 

This  report  documents  research  findings  for  the  time  period  30 
March  1970  to  19  June  1970  under  support  of  NR  276-027.   This  report 
discusses  applications  of  the  theory  of  differential  games  to  pursuit 
and  evasion  problems  of  Naval  warfare.   In  particular,  we  consider 
the  problem  of  surveillance-evasion.   A  companion  report  [47]  discusses 
other  research  findings  of  the  contract  period  with  respect  to  tactical 
allocation  problems. 

The  goal  of  this  research  is  to  determine  the  circumstances 
under  which  an  evader  can  outmaneuver  a  pursuer  to  escape  as  a  function 
of  maneuverability.   The  solution  of  this  problem  leads  to  conditions 
for  a  tracker  (destroyer)  to  be  able  to  keep  a  hostile  vehicle  (sub- 
marine) under  constant  surveillance.   The  original  approach  was  to 
survey  the  previously  published  work  in  this  field  and  to  attempt  to 
extend  these  modelling  efforts.   Detailed  analysis  of  past  work  [14] 
has  uncovered  several  flaws  in  its  mathematical  development,  and, 
hence,  the  current  work  has  concentrated  on  establishing  a  firmer 
mathematical  basis  for  the  surveillance-evasion  aspect  of  the  more 
general  problem  of  pursuit  and  evasion.   This  work  has  created  a  broad 
base  for  future  possible  extensions. 

Warfare   is  characterized  by  decisions  being  made  on  the  dynamics 
of  combat  over  a  period  of  time  by  the  antagonists  towards  conflicting 
goals.   The  creation  of  game  theory  by  J.  von  Neumann  [48],  [49] 
(although  anticipated  by  E.  Borel  [21])  has  had  a  major  impact  on  the 
modelling  of  conflict  situations.   The  optimization  of  dynamical  systems 


has  been  studied  under  the  calculus  of  variations  since  the  17th 
century.   However,  in  cases  where  inequality  constraints  are  present 
in  the  model,  these  powerful  classical  techniques  require  intricate 
modifications.   In  this  environment,  the  almost  simultaneous  develop- 
ment in  the  early  1950's  of  differential  games  by  R.  Isaacs  [26],  [27], 
[28],  [29]  and  the  Pontryagin  maximum  principle  by  the  Russian  mathe- 
matician L.  Pontryagin  [43]  has  been  enthusiastically  received  by 
military  operations  research  workers.   It  seems  appropriate  to  discuss 
these  techniques  briefly. 

a.   Differential  Games 

R.  Isaacs  was  the  original  developer  of  differential  games  in 
the  environment  of  RAND  in  the  early  1950's  [26],  [27],  [28],  [29]. 
Although  not  acknowledged  in  his  book,  he  applied  Bellman's  ideas  of 
dynamic  programming  [5]  to  a  limiting  case  of  a  multi-move   discrete 
game.   A  brief  sketch  of  the  history  of  the  later  mathematical  develop- 
ment of  differential  games  up  through  1965  is  contained  in  [22].   Isaacs 
published  a  major  work  in  1965  [30].   Y.  C.  Ho  [23]  has  reviewed  Isaacs's 
work  and  discussed  its  relationship  to  deterministic  optimal  control  theory 
in  an  excellent  review  of  the  book  Differential  Games. 

The  subject  referred  to  as  differential  games  may  in  the  future 
be  called  zero-sum  deterministic  differential  games  within  the  emerging 
framework  of  "generalized  control  theory"  [24],  [25].   It  seems  appro- 
priate therefore  to  review  briefly  the  characteristics  of  such 
optimization  models.   We  consider  two-controllers  who  manipulate  their 
own  control  variables  in  a  dynamic  system  whose  behavior  is  described 
by  a  system  of  differential  equations.   Each  controller  has  his  own 


criterion  function,  but  these  are  related  by  summing  to  a  constant. 
Hence,  one  man's  loss  is  the  other's  gain.   There  is  one  information 
set  and  it  is  perfect  in  the  sense  that  all  past  history  is  known, 
opponent's  capability,  etc.,  except  the  instantaneous  strategy  of  the 
opponent.   It  is  within  the  framework  of  these  general  assumptions 
that  idealized  surveillance-evasion  tactics  will  be  developed. 

The  work  of  Isaacs  has  been  the  major  source  of  ideas  for  the 
current  research.   Although  recent  work  has  been  more  mathematically 
precise  [6],  [7],  the  worked  examples  in  Isaacs  book  appear  to  this 
researcher  to  be  at  least  a  decade  ahead  of  the  development  of  those 
who  place  a  premium  on  precision.   Each  new  application  of  differential 
game  theory  appears  to  motivate  several  new  concepts. 

b.   Generalized  Control  Theory 

It  seems  appropriate  to  discuss  the  general  problem  of  pursuit 
and  evasion  within  the  broader  framework  of  "generalized  control  theory." 
As  noted  above,  two  notable  deficiencies  of  the  differential  game  models 
to  be  considered  in  this  work  are:   (1)  perfect  information  is  assumed 
and  (2)  the  model  is  deterministic.   Hence,  we  will  address  only  the 
kinematic  aspects  of  surveillance-evasion  and  will  not  consider  deception 
tactics . 

Within  the  past  several  years  a  probabilistic  control  theory 
has  emerged.   W.  Fleming  [19]  recently  has  reviewed  this  field  and 
provided  an  extensive  (and  basic)  bibliography.   Willman  [50]  and  Behn 
and  Ho  [4]  have  extended  these  concepts  to  conflicting  dynamical  systems. 
Such  an  approach  applied  to  the  problem  at  hand  would  consider  detection 
probabilities  and  that  the  players  have  only  imperfect  knowledge  of  the 


state  of  the  system,  i.e.,  there  is   noise   superimposed  on  the  signal 
as  to  the  location  of  one's  opponent.   Such  extensions  are  beyond  the 
scope  of  the  current  modelling  effort,  but  are  noted  for  possible  future 
extensions.   The  deterministic  model  is  complementary  to  the  stochastic 
model  and  should  provide  insight  into  the  latter. 

c.   Application  to  Problems  of  Naval  Warfare 

We  have  seen  that  differential  games  provide  a  model  for  optimizing 
conflicting  dynamical  systems  over  a  period  of  time.   There  are  numerous 
applications  of  such  models  to  problems  of  Naval  warfare: 

(1)  interception  of  enemy  missiles  by  ABM's, 

(2)  allocation  of  Naval  fire  support  to  various  targets, 

(3)  allocation  of  Naval  airpower  to  ground-support  and 
strategic  targets, 

(4)  allocation  of  effort  in  searching  for  targets, 

(5)  surveillance  and  tracking  of  hostile  vehicles. 

These  various  applications  are  noted,  since  the  solutions  of  all  these 
problems  involve  the  use  of  the  same  mathematical  technique,  differen- 
tial game  theory. 

In  the  current  research,  we  study  the  problem  of  surveillance- 
evasion,  in  which  the  "pursuer"  attempts  to  maintain  contact  with  an 
"evader"  who  attempts  to  break  contact  by  moving  outside  the  detection 
capability  of  the  pursuer.   The  mathematical  structure  of  this  problem 
is  closely  related  to  that  of  pursuit  and  evasion  problems:   in  sur- 
veillance-evasion the  problem  occurs,  for  example,  within  a  circle  and 
terminates  on  its  boundary,  while  in  pursuit  and  evasion  the  problem 
is  exterior  to  a  circle.   This  research  has  uncovered  some  subtle  differ- 
ences, however. 


The  discrete  models  of  multi-move  discrete  games  [5]  (generaliza- 
tion of  dynamic  programming)  may  be  used  to  study  these  phenomena  under 
less  restrictive  assumptions.   The  value  of  the  differential  game 
approach  is  that  it  leads  to  explicit  expressions  relating  the  various 
system  parameters.   Thus,  the  basic  structure  of  optimal  policies  and 
tradeoffs  between  system  parameters  may  be  explicitly  exhibited. 

The  mathematical  techniques  of  control  theory  show  great  promise 
for  providing  insight  into  optimizing  the  dynamics  of  Naval  warfare. 
Many  previous  analyses  involving  classical  variational  methods  may  be 
more  easily  done  and  extended  by  their  use.   As  an  example  of  this, 
de  Guenin's  extension  [13]  of  Koopman's  results  [38]  on  the  optimum 
distribution  of  searching  effort  is  derived  in  Appendix  E  by  use  of  the 
Pontryagin  maximum  principle. 


II.   REVIEW  OF  PERTINENT  LITERATURE 

The  published  literature  was  reviewed  to  find  out  what  had  been 
done  on  the  topic  of  surveillance-evasion  in  order  to  avoid  duplica- 
tion of  research  effort.   We  do  not  attempt  a  comprehensive  review  of 
the  literature,  since  that  was  not  the  purpose  of  this  research.   How- 
ever, some  major  works  are  highlighted  .   Literature  was  reviewed  in  two 
subject  areas:   search  theory  and  differential  games. 

J.  Dobbie  [15]  has  published  a  rather  comprehensive  survey  of 
search  theory  in  1968.   He  indicated  that  the  only  published  work  on 
the  tracking  operation  in  the  open  literature  was  by  Dobbie  himself 
[14].   The  1966  paper  by  Dobbie  considers  the  kinematic  aspects  of 
surveillance-evasion.   This  paper  is  the  primary  basis  for  the  present 
research.   Dobbie  considers  a  sequence  of  problems,  formulated  as  differ- 
ential games:  surveillance-evasion  for  a  circular  detection  region  of 
the  pursuer,  tracking  for  an  arbitrary  detection  region,  and  two  models 
in  which  recontact  is  possible  by  the  pursuer.   The  content  of  this  paper 
has  evolved  into  operational  Navy  doctrine  [16]. 

A  more  extensive  search  of  the  literature  did  not  yield  any  further  work 
on  tracking  operations  in  the  open  literature.   The  1966  survey  by 
Enslow  [18]  and  S.  Pollock's  selected  bibliography  [42]  were  consulted 
in  this  respect.   Both  these  surveys  were  consulted  by  Dobbie  for  his 
survey  article  [15].  A  recent  effort  at  the  University  of  Michigan  [41] 
was  also  examined  and  did  not  yield  any  new  references  on  the  tracking 
problem. 

The  differential  game  literature  was  consulted  for  general  mathe- 
matical background  and  to  see  if  applications  to  tracking  could  be  found. 


The  applications  literature  was  considered  of  prime  importance  rather 
than  the  development  of  the  mathematical  theory.   Isaacs' s  1965  book 
[30]  remains  the  chief  source  of  examples  and  insight  into  technical 
questions  related  to  solving  actual  problems.   His  terminology  is  non- 
standard to  control  theorists,  little  use  or  reference  to  the  classical 
variational  methods  is  made,  but  he  does  provide  an  extensive  theory 
naturally  motivated  through  examples.   Isaacs  homicidal  chauffer  problem 
is  basic  to  pursuit  and  evasion  studies.    He   had  also  considered 
the  tracking  problem  while  at  CNA  [31].   This  reference  does  not  contain 
any  analysis  and  the  condition  developed  for  surveillance  to  be  main- 
tained is  incorrect  (as  is  that  developed  by  Dobbie  [14]).   Isaacs' s 
sketches  of  the  problem  indicate  that  he  did  note,  however,  the  termina- 
tion of  the  barrier. 

L.  Berkovitz's  paper  [6]  presents  an  extremely  rigorous  mathe- 
matical development  of  differential  game  theory  but  presents  no  examples 
and  does  not  consider  most  of  the  significant  aspects  required  to  solve 
specific  problems.   Blaquiere,  Gerard,  and  Leitman  [8]  have  recently 
published  a  book  on  differential  games.   This  book  develops  the  theory 
from  a  geometric  point  of  view  and  is  an  extension  of  the  geometric 
approach  to  control  theory  problems  developed  by  these  authors  over  the 
past  five  years.   This  previous  work  is  accessable  through  the  biblio- 
graphy in  this  book.   Although  some  examples  are  given,  they  don't 
appear  to  be  representative  of  a  broad  experience  in  applications  as 
is  the  case  of  those  in  Isaacs'  book.   This  book  is  not  written  for  one 
not  already  acquainted  with  the  theory  and  is  not  useful  for  the  novice. 


The  intimate  connection  between  differential  games  and  control 
theory  is  pointed  out  in  numerous  places  (see,  for  example  [6],  [23], 
[30]).   In  view  of  this  relationship,  much  of  the  optimal  control  theory 
can  be  brought  to  bear  on  differential  game  problems.   An  excellent 
review  of  deterministic  optimal  control  theory  with  an  extensive 
bibliography  is  by  Athans  [1].   An  excellent,  concise  discussion  of  the 
relationships  between  control  theory  and  differential  games  is  contained 
in  the  review  of  Isaacs  book  by  Y.  C.  Ho  [23].   Other  articles  which 
contain  useful  review  material  are  by  Ho,  Bryson,  and  Baron  [22]  and 
also  Sarma  and  Ragade  [45].   Athans  and  Falb  [2]  have  written  an  excellent 
introductory  text.   The  excellent  book  by  Bryson  and  Ho  [9],  besides 
being  an  easily  understood,  lucid  introduction  to  the  field,  contains 
many  advanced  topics  including  a  brief  introduction  to  differential 
games . 

The  Russians  have  done  extensive  research  on  optimal  control/ 
differential  games  over  the  last  decade  [3].   An  excellent  survey  article 
delineating  numerous  fields  of  application  and  with  an  extensive  biblio- 
graphy of  original  Russian  research  papers  is  by  Simakova  [46].   Y.  C. 
Ho,  one  of  the  best  qualified  individuals  to  survey  the  current  Western 
state-of-the-art,  has  documented  current  developments  (theoretical  and 
in  applications)  [25].   An  important  question  in  pursuit  and  evasion 
problems  is  "When  can  capture  occur?"   Isaacs  [30]  has  developed  several 
(equivalent)  criteria  for  determining  the  useable  part  of  the  terminal 
surface.   Recently,  L.  Meier  [40]  has  proposed  a  new  geometrical  criterion 
from  his  study  of  ABM  interception  of  re-entry  vehicles. 


III.   The  Surveillance-Evasion  Problem 

We  consider  an  idealized  model  for  the  problem  of  a  tracker 
(for  example,  a  destroyer)   keeping     a  hostile  vehicle  (for  example, 
a  submarine)  under  constant  surveillance.   The  maneuverabilities  (i.e., 
maximum  speed  and  turning  capabilities)  of  the  two  vehicles  lead  to  the 
appropriate  conditions.   When  these  conditions  are  met,  the  pursuer  can 
keep  the  Evader  under  close  surveillance.   The  Evader  cannot  do  anything 
to  break  contact  and  prevent  this  "tracking"  or  "tailing."   We  consider 
an  extremely  idealized  model  with  perfect  information  on  the  location 
of  the  enemy  for  both  antagonists  and  a  circular,  "cookie-cutter" 
detection  region  for  the  Pursuer. 

a.   Statement  of  the  Problem. 

We  consider  the  same  model  used  by  Dobbie  [14]  ,  which  is  an  exten- 
sion of  Isaacs 's  model  [30].   There  are  several  flaws  in  the  earlier 
mathematical  development  of  Dobbie,  which  lead  to  an  incorrect  condition 
for  surveillance  to  be  maintained  and  an  incorrect  analysis  for  surveil- 
lance-evasion with  arbitrary  detection  regions.   The  latter  has  the 
important  implication  that  the  involute  tactic  [16]  may  not  be  optimal 
for  holding  contact  in  real  world  situations  where  sonar  capabilities 
generate  a  non-circular  detection  region.   It  should  also  be  noted  that 
the  brief  work  by  Isaacs  on  this  problem  also  yielded  the  wrong  surveil- 
lance condition.   Hence,  the  purpose  of  the  present  research  is  to  set 
such  analysis  on  a  firmer  mathematical  basis. 

In  the  model  the  Pursuer  is  faster  than  the  Evader,  who  does, 
however,  possess  an  advantage  in  turning  capability.   We  assume  a 
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circular  "cookie-cutter"  detection  region  for  the  Pursuer,  who  detects 
the  Evader  with  probability  one  when  they  are  less  than  a  distance   d 
apart.   The  goal  of  the  Evader  is  to  break  contact  as  quickly  as  possible 
by  moving  out  of  the  circular  detection  region  of  the  Pursuer,  who 
attempts  to  maintain  contact  as  long  as  possible.   We  define  the  follow- 
ing notation: 

subscripts:   1  refers  to  Pursuer,   2  refers  to  Evader 

s..  =  Pursuer's  speed  with  maximum  w 

s„  =  Evader's  speed  with  maximum  w 

R  =  minimum  turning  radius  of  the  Pursuer 

<j>   =  fraction  of  maximum  course  curvature  employed  by  Pursuer 

(<j>  =  -1   corresponds  to  left  turn  with  minimum  turning  radius) 

ii     =   Evader's  heading  relative  to  that  of  Pursuer 

d  =  radius  of  Pursuer's  detection  region 

T  =  time  for  Evader  to  escape  (reach  circle  of  radius  d   from 
Pursuer) 

Thus,  the  problem  facing  the  Pursuer  is 

T 
max  min   dt , 
0 

subject  to  the  equations  of  motion  of  the  two  vehicles.   It  is  conven- 
ient to  adopt  a  relative  coordinate  system  for  two  reasons:   (1)  the 
dimension  of  the  problem  is  reduced  and  (2)  such  a  coordinate  system 
is  standard  in  Naval  operations.   In  this  relative  system  the  Pursuer 
is  located  at  the  origin  and  has  a  verticle  heading.   Thus  the  coordinate 
system  is  "carried  by  the  Pursuer."   In  this  system  the  problem  is 
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max        mm 
*>S1   ^'S2    0 


dt      with      T      unspecified, 


dx      V* 


subject  to:   -7""  =  -  ~ —  +  s.  sin  \b. 
dt      R      2 


dy    S1X* 

dt  =  ~r~  +  s2  cos  *  "  V       (1) 


and   -1  s  f  s  1, 


0  £  s   £  w  , 


0  £  s  £.   w  <  w  , 

with  initial  location  of  evader 

x(t  =  0)  =  x  , 

y(t  =  0)  =  yQ, 

and  terminal  surface  defined  by 

x2(T)  +  y2(T)  =  d2. 

These  equations  are  derived  in  Appendix  A,  since  the  solution  procedure 
relies  on  an  understanding  of  the  geometry  of  this  relative  coordinate 
system. 

b.   Games  of  Degree  and  Games  of  Kind. 

In  this  section  we  discuss  the  concepts  of  a  game  of  degree  and 
a  game  of  kind.   The  purpose  of  this  discussion  is  to  explain  that  in 
order  to  solve  a  game  of  kind  one  must  solve  that  part  of  a  correspond- 
ing game  of  degree  into  which  the  game  of  kind  has  been  imbedded.   The 
common  part  of  optimal  strategies  for  these  two  games  is  the  barrier, 
i.e.  boundary  of  domain  of  controllability. 
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Isaacs  ([30]  p.  35)  defines  a  game  of  degree  as  one  with  a  contin- 
uous range  of  payoffs.   In  the  surveillance-evasion  game,  the  formulation 
of  the  previous  section  is  a  game  of  degree  with  time  of  escape  as  the 
payoff.   A  game  of  kind  is  one  with  a  finite  number  of  payoff  values.   The 
reaching  of  each  particular  terminal  state  yields  a  single  payoff  value. 
Such  is  the  case  if  we  consider  the  payoffs  as   +1   if  contact  is  broken 
and   -1   if  the  Evader  can't  break  contact.   Blaquiere  et  al  ([8]  pp.  9-10) 
have  a  similar  definition  except  that  a  quantitative  game  (game  of  degree) 
is  a  game  in  which  there  is  a  common  target  set  for  both  pursuer  and 
evader.   It  should  be  noted  that  there  are  attrition  games  in  the  liter- 
ature for  which  this  definition  is  inadequate  [47]. 

We  define  the  domain  of  controllability  of  a  terminal  state  to  be 
that  subset  of  the  initial  state  space  from  which  trajectories  lead  to 
this  terminal  state  for  all  admissible  strategies  of  the  player  for 
which  this  terminal  state  is  unfavorable  when  the  extremal  strategy  of 
his  adversary  is  played.   By  an  extremal  strategy,  we  mean  a  strategy 
determined  on  an  extremal  trajectory,  which  is  a  path  on  which  the 
necessary  conditions  for  optimality  [6]  are  almost  everywhere  satisfied. 
The  barrier,  being  the  boundary  of  the  domain  of  controllability,  is 
the  trajectory  which  leads  to  the  boundary  of  the  useable  part  of  the 
terminal  surface.   The  useable  part  of  the  terminal  surface  is  that  part 
of  the  terminal  surface  to  which  there  are  paths  from  the  state  space 
(see  Issacs  [30]  p.  83  and  also  Appendix  B) . 

Thus  Isaacs  ([30]  p.  13)  imbeds  a  game  of  kind  into  a  game  of 
degree.   We  may  consider  a  game  of  degree  as  a  game  of  kind  in  which 
the  prerogative  of  each  of  the  antagonists  is  exercised  to  do  his  "best." 
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The  point  being  developed      Is  that  one  must  know  how  to  solve  a 
game  of  degree  in  order  to  solve  the  corresponding  game  of  kind.   Other 
uses  of  the  game  of  degree  solution  are 

(1)  show  path  of  system  when  prerogatives  are  exercised, 

(2)  when  the  prerogative  is  not  exercised  by  one  party,  indicates 
direction  system  will  move. 

The  latter  remark  is  the  motivation  for  Isaacs' s  concept  of  a  semi- 
permeable surface  ([30]  pp.  70-71). 

Let  us  discuss  further  the  concept  of  a  semi-permeable  surface, 
since  even  one  of  the  leading  control  theorists  has  overlooked  the 
reason  for  this  concept  in  his  excellent  review  of  Isaacs' s  book  [23]. 
Initially,     we  consider  a  game  of  degree  and  a  simply-connected 
domain  in  the  state  space.   When  the  problem  has  a  solution,  this 
domain  is  covered  by  a  field  of  extremals  [17] .   We  can  transform  our 
original  problem  to  one  with  terminal  payoff,  so  that  the  value  of  the 
game,  denoted  by  V,  will  be  constant  along  an  extremal.   Consider  now 
Figure  1.   Three  extremals  with  the  accompanying  value  of  the  game  are 
shown.   If  extremal  strategies  are  used  by  both  players,  the  trajectory 
remains  on  the  extremal.   If  the  Pursuer  uses  his  optimal  strategy   <J>  , 
but  the  Evader  uses  a  non-optimal  strategy,  then  the  course  of  the 
system  is  steered  to  lower  payoff  values  and  similarly  when  the  roles 
are  interchanged. 

To  reiterate,    the  semi-permeable  surface  is  a  surface  in  the 
state  space  fcr  which  each  player  controls  the  penetration  by  the  path 
of  the  system  by  use  of  his  optimal  strategy.   When  a  player  employs 
his  optimal  strategy,  his  opponent  can't  steer  the  system  to  a  more 
favorable  position  by  any  strategy,  and  if  he  doesn't  use  his  own  optimal 


14 


V 


V  =  10 


*      V  =  ' 


x 


\ 


<j)  -  Pursuer  (minimizes) 
\\)  -   Evader  (maximizes) 


Figure  1.   Semi-permeable  Surface. 
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strategy,  a  less  favorable  outcome  will  result.   This  situation  is  a 
consequence  of  the  criterion  functional  having  a  saddle  point  when 
optimal  strategies  are  used.   The  purpose  of  Isaacs'  introduction  of  this 
concept  of  a  semi-permeable  surface  is  that  the  barrier,  the  only  place 
where  optimal  strategies  are  determined  for  a  game  of  kind,  is  a  semi- 
permeable surface. 

The  barrier  is  a  surface  in  the  state  space  (Isaacs  refers  to  the 
state  space  as  the  "playing  space")  which  divides  the  capture  and  escape 
zones  when  both  exist.   The  global  answer  to  the  capture-or-escape  ques- 
tion depends  on  whether  or  not  the  barrier  divides  the  state  space  into 
two  parts.   Hence,  we  know  that  if  we  can  show  that  the  barrier  termi- 
nates without  having  done  this,  the  entire  state  space  is  either  all 
capture  zone  or  all  escape  zone.   For  the  problem  at  hand  (surveillance- 
evasion)  this  means  that  there  is  probably  no  surveillance  zone  if  the 
barrier  so  terminates.   Termination  of  the  barrier  is  discussed  in 
Isaacs's  book  [30]  on  pages  210-214.   The  argument  given  there  that  a 
barrier  terminates  due  to  an  abrupt  change  in  direction  (see  Figure  2) 
is  best  understood  by  recalling  that  a  game  of  kind  is  imbedded  into  a 
game  of  degree.   In  this  game  of  degree,  each  behavior  would  lead  to 
more  than  one  extremal  at  a  point  in  the  state  space, 
c.   Connection  with  the  Homicidal  Chauffer  Game 

The  research  philosophy  has  been  to  employ  a  "broad"  approach  to 
specific  problems  by  drawing  upon  theory  from  diverse  fields.   For 
example,  consideration  of  the  geometric  properties  of  complex  numbers 
has  led  to  a  geometric  way  to  construct  extremal  paths  in  the  homicidal 
chauffer  game.   This  led  to  the  correct  condition  for  termination  of  the 
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Figure  2.   Conditions  for  Termination  of  Barrier. 
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barrier  (and  hence  the  existence  of  no  surveillance  region) ,  which  had 
eluded  both  Dobbie  [14]  and  Isaacs  [31]. 

Consideration  of  a  sequence  of  closely  related  problems  has  been 
attempted    to  try  to  learn  from  their  common  points.   Three  such 
closely  related  problems  are: 

(1)  destroyer  to  fixed  destination, 

(2)  homicidal-chauffer  game, 

(3)  surveillance-evasion  game. 

In  working  on  the  surveillance-evasion  game,  consideration  of  the  first 
two  problems  has  proven  to  be  useful.   The  study  of  these  has  many 
points  in  common  with  the  surveillance-evasion  game.   Hence,  analysis 
of  these  problems  is  presented  in  Appendices  B  and  C  as  background 
material. 

Specifically,  the  relationship  of  the  homicidal-chauffer  game  and 
the  surveillance-evasion  game  is  as  follows  (as  first  pointed  out  by 
Isaacs  [31]).   Consider  the  terminal  surface  of  the  homicidal-chauffer 
game.   It  is  a  circle,  and  the  state  space  for  the  game  is  exterior  to 
this  circle.   For  the  surveillance-evasion  game,  the  state  space  is 
interior  to  the  terminal  surface  with  the  useable  and  non-useable  parts 
of  the  terminal  surface  being  interchanged.   A  point  worth  noting  (again 
first  pointed  out  by  Isaacs  [31])  is  that  most  pursuit  differential  games 
can  be  converted  into  surveillance  games  by  turning  the  analysis  inside 
out. 

d.   Solution  of  the  Surveillance-Evasion  Game 

In  this  section,  we  show  the  solution  to  the  game  of  degree.   Analysis 
details  are  presented  in  Appendix  D.   As  noted  in  Section  Illb ,  we  may 
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consider  the  game  of  kind  as  imbedded  in  a  game  of  degree.   Hence,  when 
we  consider  the  problem  as  a  game  of  kind  (Is  it  possible  for  surveil- 
lance to  be  maintained?),  optimal  strategies  of  the  game  of  degree  only 
apply  on  the  barrier  for  the  game  of  kind. 

Before  discussing  briefly  the  geometric  aspects  of  solution,  let  us 
summarize  the  new  results  of  the  current  research: 

(1)  correct  condition  for  surveillance  to  be  maintained, 

(2)  new  geometric  construction  for  escape  paths, 

(3)  extension  of  model  to  non-circular  detection  region. 
An  important  question  to  be  answered  by  the  model  is,  "Under  what 
circumstances  can  surveillance  be  maintained?"   The  correct  answer  is 
when 


d  ;>  R{  /l  -(w2/w1)^  +  2  (w2/w1)  (tt  -  U) } 


(2) 


where 


cos  U  =  w  /w,   and   0  £  U  ^  tt/2. 


Dobbie  [14]  and  Isaacs  [31]  had  derived  different  conditions. 

We  also  have  discovered  a  geometric  interpretation  for  the  optimal 
escape  paths.   For   0  £  t  £  2R/w  (tt  -  u)  ,   the  path  equation  may  be 
written  as 


x(t)-  R 
yd) 


cos  w  t/R    sin  w  t/R 
-sin  w   t/R    cos  w   t/R 


(d   -    t   w   )sin   u  -  R 
(d   -   t   w   )cos   u 


(3) 


where       U    £  u  £  tt/2      and      cos  U   =  w9/w-i  >  0  £  U   £  tt/2 


t  =  T  -  t 


i.e.,  time  measured  backwards  from  escape. 
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This  equation  says  that  the  Evader's  location  on  an  optimal  escape  path 
may  be  obtained  as  follows  (with  reference  to  Figure  3) : 

(1)  locate  escape  point  on  detection  circle,  for  example  A  , 

(2)  point  moves  along  line   OA   with  speed  w   towards   0; 
at  time   t   it  is  at   A„  , 

(3)  rotate  A   through  angle  w1  t/R  in  negative  sense  about 
point   (R,0);   this  yields  a',   the  point  on  the  optimal 
trajectory  at  time   x, 

(4)  maximum  possible  rotation  is  through  angle   2(tt  -  u)  ,   where 
u  is  angle  between  line  from  escape  point  on  detection 
circle  to   0  and  the  positive  y-axis. 

At  a  later  time   t„,   the  point  has  moved  to  A„   and  is  rotated  to 
A'   on  the  escape  path.   When  the  optimal  trajectory  is  the  barrier, 
there  is  an  additional  special  geometric  property  of  the  escape  path. 
It  may  be  shown  that  the  barrier  is  an  involute  to  a  circle  of  radius 
R  w  /w   with  center   (R,0).   Hence,  at  each  point  on  this  curve,  the 
normal  is  tangent  [12]  to  the  circle  just  mentioned.   This  is  illustrated 
for  point  A'   in  Figure  3.   For  non-barrier  escape  paths  (consider  path 
B  B' B' )  ,   the  curve  is  not  an  involute  to  any  circle,  but  the  same  geo- 
metric construction  holds. 

This  geometric  construction  provides  deeper  insight  into  the 
geometry  of  escape  paths.   (A  similar  geometric  construction  is  possible 
for  both  the  destroyer  to  fixed  destination  problem  and  the  homicidal 
chauffer  game.)   It  suggests  that  there  may  be  a  conjugate  point  [44] 
to  the  escape  point  on  an  optimal  trajectory,  i.e.,  neighboring  extremals 
intersect  at  this  point.   Such  a  point  is   B'   in  Figure  3.   Further 
investigation  of  this  phenomena  seems  warranted  but  time  hasn't  permitted 
it.   The  geometric  construction  suggests  that  the  problem  may  be  more 
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Figure  3.   Geometric  Construction  for  Optimal  Escape  Paths. 
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easily  solved  in  relative  polar  coordinates,  but  this  has  not  been 
explored  too  far  as  yet.   A  similar  construction  results  when  the  solu- 
tion is  extended  past  the  time  restriction  above.   We  note  that  the 
y-axis  is  a  line  of  symmetry  for  this  problem. 

We  have  briefly  examined  this  problem  for  a  non-circular  detection 
region.   Our  research  indicates  that  Dobbie's  [14]  approach  is  incorrect 
and  that  the  problem  should  be  re-examined  for  some  simple  non-circular 
geometries.   This  implies  that  the  involute  tactic  [16]  may  not  be 
optimal  for  non-circular  detection  regions.   The  justification  for 
these  statements  is  given  in  the  next  section. 

We  now  consider  the  geometry  of  the  solution  to  the  tracking  pro- 
blem.  We  shall  describe  the  optimal  trajectories  (as  far  as  the  current 
research  has  progressed)  and  optimal  tactics.   The  type  of  geometric 
configuration  for  the  escape  paths  depends  on  the  craft  speeds,  the 
Pursuer's  minimum  turning  radius,  and  the  radius,   d,   of  his  detection 
region.   We  let 


d1   =  R{/1  -(w2/Wl)z  +  2(w2/w1)(tt   -U)},  (4) 

d2    =    R{/1    -(w2/Wl)Z    +(W2/W1)(    |    7T    -U)+    l},  (5) 


where 


cos   U  =  w   /w        and      0  £  U  £  tt/2. 

Then,  there  are  three  cases  (see  Appendix  D  for  details): 

(1)  d  <  d  no  surveillance  region  (of  type  in  next  two  cases) 

(2)  d±   s  d  <  d2 

(3)  d?  £  d         barrier   meets  the  negative  y-axis. 
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These  three  cases  are  shown  in  Figures  4  through  6.   In  all  cases  the 
barrier  is  the  involute  to  a  circle  of  radius  R  w„/w1   and  center 
(+  R,0).   The  tangents  to  these  circles  from  0  are  of  special  signi- 
ficance.  The  escape  paths  are  symmetric  with  respect  to  the  y-axis , 
and  hence  we  discuss  the  solution  only  in  the  right  half-plane.   Let 
u  be  the  angle  between  line  from  escape  point  on  detection  circle  to 
0   and  the  positive   y-axis.   The   equation  of  escape  paths  is  given 
by  (3)  for  range  of   x   given  there. 

In  case  (1) ,  the  barrier  terminates  when  it  reaches  the  circle 
of  which  it  is  the  involute  (see  AA'  in  Figure  4).   Paths  which  terminate 
between  A  and  B  are  given  by  equation  (3),  but  it  hasn't  been  ascer- 
tained whether  such  paths  terminate  abruptly  (yielding  a  dispersal  surface) 
for  x  £  x   =2  R/w  (tt  -  u)   due  to  intersection  of  neighboring  extremals. 
Escape  paths  have  not  been  traced  backwards  for  t  ^  x, .   Thus,  although 
there  is  no  surveillance  region  of  the  type  for  cases  (2)  and  (3) ,  the 
optimal  escape  paths  have  not  been  determined  from  all  of  the  state  space, 
and  there  may  be  "surveillance  pockets"  present.   Hence,  we  have  estab- 
lished the  Evader's  escape  tactics  only  in  the  small  region  AA'CB,   when 
he  is  close  to  the  limits  of  the  Pursuer's  detection  capability.   One 
disturbing  feature  of  this  model  is  that  for  escape  at   x(T)  =  d  sin  u 
and   tt/2  <  u  <  3/2  tt  ,   the  Pursuer's  optimal  tactic  is  to  stop  dead  in 
the  water,   s  =  0.   Such  escape  paths  (shown  in  Figure  4)  terminate 
at  D,  E,  and  F  and  originate  from  0. 

In  case  (2),  the  barrier  divides  the  state  space  into  a  surveillance 
zone  and  an  escape  zone.   For  0  £  x  £  x   =2  R/w  (tt  -  ^)  ,   the  Pursuer 
uses   <j>  =  1   (sharpest  turn  to  right).   For  x  >  x  ,   the  Pursuer  uses 
<J>  =  -1,   and  the  barrier  is  given  by 
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Figure  4.   Geometry  of  Tracking  Problem,   d  <  d-^ 
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'x(t)+R 

.  y(t)  . 


cos  w1(t-t.)/R  -sin  w.Ct-t.J/R 
sin  w  (t-t..)/R   cos  w  (t-t..)/R 


'x(t  )+w  (t-t  )cos(Tf/2-'U)+R 
ky(t  )-w2(t— r  )sin(Tr/2-U) 


(6) 


which  may  also  be  shown  to  be  an  involute  unwinding  from  a  circle  with 

center  at   (-R,0)   and  radius   R  w_/w  .   At   x  =  t  ,   the  point  D   is 

always  located  on  the  lower  tangent  to  the  circle  as  shown  in  Figure  5. 

It  lies  between   0  and  D'   (when  d  =  d  ,   it  is  at  D').   The  paths 

terminating  between  A  and  B   intersect  each  other,  but  complete 

details  have  not  been  worked  out  as  yet.   Again,  for  escape  at   x(T)  =  d  sin  u 

and  tt/2  <  u  <  3/2  it,   the  escape  paths  are  straight  lines  terminating 

at,  for  example,   F,  G,  H,  and  I.   Paths  in  the  vicinity  of  ODE  (a  "pocket") 

haven't  been  worked  out. 

In  case  (3),  the  Pursuer  only  uses   <J>  =  1  on  the  barrier,  since 
it  intersects  the  negative  y-axis  before   t  =  x_ .   When  d  =  d  ,   the 
barrier  is  tangent  to  the  negative  y-axis.   Other  aspects  are  similar 
to  above.   This  case  is  shown  in  Figure  6. 

e.   Shortcomings  of  Previous  Work 

The  differential  game  solution  techniques  of  the  current  research 
differ  from  Dobbie's  approach  [14].   It  is  the  purpose  here  to  discuss 

such  differences,  since  some  of  our  results  differ  from  his.   Dobbie, 

considering  a  game  of  kind,  uses  Isaacs 's  "game  of  kind  approach," 

(see  chapter  8  in  [30]).   As  we  have  discussed  above,  Isaacs  developed 

this  approach  by  imbedding  such  a  problem  in  a  game  of  degree.   The 

analytic  details  of  many  steps  in  the  solution  of  a  game  of  kind  (see 

[30]  pp.  205-210)  will  be  seen  to  be  the  same  as  employed  here  (game 

of  degree)  with  the  vector  of  dual  variables   p   replaced  by  the  normal 

to  the  barrier   v.   The  Hamiltonian  is  also  modified  slightly.   However, 
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Figure  5.   Geometry  of  Tracking  Problem,   d  £  d  <  d  . 
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Figure  6.   Geometry  of  Tracking  Problem,  d^   £  d. 
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two  important  aspects  are  not  adequately  treated  in  Dobbie's  work  [14]: 

(1)  termination  of  the  barrier  (see  [30]  pp.  210-214), 

(2)  construction  of  the  barrier  (see  [30]  pp.  214-215). 

As  we  have  discussed  above  in  section  Illb  (see  also  Appendix  B) , 
whether  or  not  there  is  a  surveillance  region  is  dependent  upon  whether  the  barrier 
terminates  or  not.   This  aspect  of  solution  is  ignored  by  Dobbie,  whose 
condition  for  the  existence  of  a  surveillance  zone  should  be  contrasted 
with  ours. 

A  more  serious  criticism  must  be  leveled  at  Dobbie's  method  of 
determining  the  useable  part  of  the  terminal  surface,  denoted  as  UP. 
Dobbie  does  not  make  use  of  the  fact  that  every  game  of  kind  is  imbedded 
in  a  game  of  degree.   Hence,  he  does  not  recognize  that  the  solution 
depends  on  the  geometry  of  the  detection  region  and  erroneously  concludes 
that  the  solution  for  a  circular  detection  region  would  apply  for  an 
arbitrary  detection  region  ([14]  p.  177).   The  purpose  of  criticism 
of  Dobbie's  results  is  to  point  out  that  the  surveillance-evasion  problem 
has  not  been  solved  for  arbitrary  detection  regions  and  suggest  such  a 
task  as  a  future  research  effort. 

Let  us  discuss  why  the  solution  depends  on  the  geometry  of  the 
detection  region.   Isaacs  ([30]  p.  215)  states  a  criterion  for  the 
construction  of  the  barrier:   the  normal  to  the  barrier  coincides  with 
the  normal  to  the  terminal  surface.   This  leads  to  our  major  criticism 
of  this  pioneering  effort:   Dobbie  tried  to  extend  the  model's  solution 
for  a  circular  detection  region  to  arbitrary  detection  regions  when 
such  an  extension  is  not  justified. 
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We  consider  a  second  argument  (from  optimal  control  theory)  for 
solution  dependence  on  geometry  of  detection  region.   Dobbie  (p.  174) 
says,  "Let   t  =  0  at  a  point  A  of  a  barrier  for  which  yv  =  xv  , 
so  that  the  normal  lies  along  the  radial  line  OA."   Since   t  =  0   is 
time  of  escape,  the  barrier  is  tangent  to  the  detection  region  at 
escape  (normal  of  barrier  is  perpendicular  to  escape  surface)  for  a 
circular  detection  region  but  does  not  have  to  be  for  an  arbitrary 
detection  region.   It  is  well-known  in  control  theory  (see  [2]  p.  290) 
that  p   is  parallel  to  n  where  p   is  vector  of  dual  variables  at 
terminal  surface  and  n  is  normal  to  terminal  surface  (pointing  inward 
to  state  space).   For  the  problem  at  hand,  the  normal  to  the  barrier 
(in  Dobbie 's  notation)  is  parallel  to  p.   Hence,  this  normal  must  be 
perpendicular  to  the  detection  region  at  the  moment  of  escape,  and  this 
condition  may  be  violated  in  Dobbie' s  analysis  for  an  arbitrary  detection 
region. 
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IV.   Conclusions  and  Future  Extensions 

Here  we  summarize  what  we  have  done  and  suggest  possible  future 
research.   We  think  that  we  have  established  more  firmly  the  mathematical 
basis  of  a  certain  type  of  surveillance-evasion  model.   Specifically, 
we  have  accomplished  the  following: 

(1)  parts  of  the  surveillance-evasion  game  of  degree  have  been 
solved  (A  disturbing  aspect  is  that  the  Pursuer  stops  still 
in  the  water  for  those  cases  when  the  Evader  escapes  "behind 
him."). 

(2)  correct  condition  developed  for  surveillance  zone  to  exist, 

(3)  devised  geometric  construction  for  describing  optimal  escape 
paths , 

(4)  showed  that  Dobbie's  extension  of  the  solution  to  arbitrary 
detection  regions  was  incorrect. 

Based  on  this  research  effort  we  suggest  the  following  as  possible  future 

work: 

(1)  develop  further  the  solution  to  the  game  of  degree  (This 
would  provide  insight  into  Evader  escape  paths  and  tactics, 
especially  for  those  cases  when  contact  can  be  broken,  i.e., 
no  surveillance  zone.), 

(2)  examine  problem  for  non-circular  detection  regions  (This 
would  allow  actual  sonar  patterns  to  be  more  accurately 
described  in  the  model.   We  suggest  that  analysis  first 
consider  an  elliptical  detection  region  and  then  try   to 
generalize  results.), 

(3)  consider  surveillance-evasion  game  in  relative  polar 
co-ordinates  (This  approach  is  suggested  from  new  geometrical 
construction  noted  above  and  is  related  to  tasks  (1)  and 

(2)  above.), 

(4)  study  extensions  of  basic  surveillance-evasion  scenario, 

(a)  formulate  problem  which  eliminates  a  stationary  Pursuer 
as  an  optimal  tactic, 

(b)  study  effects  of  Evader  having  maneuverability  limita- 
tions (What  quantitative  effect  does  this  have  on 
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condition  for  Pursuer  to  have  surveillance  zone?   This 
would  be  an  application  of  the  extension  of  Isaac's 
homicidal  chauffer  game  called  the  game  of  two  cars 
[30]  p.  237.), 

(c)  study  problem  of  two  Pursuers  against  a  single  Evader, 

(d)  develop  other  models  of  tactical  interest  and  study 
other  extensions  in  the  literature. 
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APPENDIX  A.   DERIVATION  OF  BASIC  EQUATIONS. 

In  this  appendix  we  derive  equations  (1)  of  the  main  text.   Even 
though  these  equations  are  briefly  derived  on  p.  30  of  [30],  we  feel 
that  the  current  derivation  has  increased  our  understanding  of  the 
relative  coordinate  system  and  may  be  useful  to  others.   First  we 
translate  the  restriction  of  a  finite,  non-zero  minimum  turning  radius 
into  a  restriction  of  the  maximum  rate  of  change  of  direction.   Next, 
we  develop  the  basic  equations  in  a  fixed  reference  frame.   Finally, 
the  equations  are  transformed  to  the  relative  coordinate  system. 

a.   Implication  of  Finite,  Non-zero  Minimum  Turning  Radius. 

We  consider  motion  of  a  point  in  the  plane  when  the  radius  of 
curvature  is  bounded  below  and  greater  than  zero.   We  assume  that  the 
point  is  moving  with  constant  speed  w  and  adopt  a  fixed  rectangular 
frame  of  reference  as  shown  below.   The  curve  is  given  parametrically 
by 
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x  =  x(t)   and  y  =  y(t),   where   t   is  time.   Hence,  the  velocity 
components  are  given  by 


dx 
dt 


=  w  cos  6 , 


(Al) 


dy_ 

-r-   =  w  sin 

at 


If  there  were  not  restrictions  on  maneuverability,  then  we  could  choose 

9  =  9(t)   in  a  completely  arbitrary  manner.   Let  us  now  see  how  a  lower 

d9 


bound  on  the  turning  radius  restricts   9  =  9(t)   through 


dt* 


We  consider  a  curve  in  the  plane  given  by  x  =  x(s)   and  y  =  y(s), 
where  the  parameter,   s,   is  arc  length.   The  curvature ,   k,  is  defined 
as  the  rate  of  change  of  angle  of  inclination,   9,   with  respect  to  arc 
length  (see  pp.  280-282  in  [12]) 


1  .   A9    d9 
k  =   lim  —  =  — , 
.   _  As    ds 
As->0 


where  the  angle  of  inclination  is  shown  in  the  figure  below  and  the 


slope  of  the  curve  is  related  to   9  by  tan 


_  dy. 
"  dx* 
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The  radius  of  curvature  of  the  curve  at  a  point  is  defined  by  p  =  l/< 
The  restriction  that  the  radius  of  curvature  is  bound  below  may  be 

expressed  as   p  ^  R  >  0  or   k  £  1/R.   Introducing  a  new  "turning" 
variable,  we  may  write  this  as  an  equality 

<  =  4>/R  where   -1  £  <J>  £  1.  (A2) 

Since  time  is  a  more  convenient  parameter  than  arc  length,  we  have 

d0    d6    dt    d6  ,ds  /AON 

K  =  T~  =  IT  '  ~7~  =   TT'77-  (A3' 

ds    dt    ds    dt  dt 


Recalling  that   ds  =  /ds2  +  dyz,   we  have  that 

X 


ds 
dt 


-  fW^W  -  •■  ^ 


by  use  of    (Al).      Combining    (A2) ,    (A3),    and    (A4),   we   obtain 

—  =   4>w/R     where     -1  S  |  S  1, 

which  is  the  desired  restriction  on   6(t)   from  the  lower  bound  on 
the  "turning"  radius.   We  summarize  the  equations  of  motion  in  a  plane 
when  there  is  a  lower  bound  on  the  radius  of  curvature,   R. 

dx  A 

3—  =  w  cos  9, 

dt 

dy 

-?-   =  w  s  in  0 , 

at 

—  =  <j)w/R  where  -1  £  <f>  £  1.  (A5) 

b.   The  Relative  Coordinate  System. 

It  is  convenient  to  adopt  a  relative  coordinate  system,  one  that 
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moves  with  the  observer.   Our  development  is  the  same  as  Koopman's 

([38]  or  again  [39]).   We  consider  first  the  simple  case  of  motion  at 

fixed  speed  and  course  for  an  observer  (who  carries  the  coordinate 

system  with  him)  and  a  target.   Assuming  both  the  observer  and  the 

target  move  at  constant  speeds  in  straight  lines,  we  let 

v  =  speed  of  observer  in  knots  (ocean  or  true  speed) , 

u  =  speed  of  target  in  knots  (ocean  or  true  speed) , 

w  =  speed  of  target  relative  to  observer  in  knots. 

The  corresponding  vectors,   v,   u,   and  w,   describe  the  constant 

motion.   We  note  that  the  vector  difference,   w  =  u  -  v,   gives  the 

motion  of  the  target  relative  to  that  of  the  observer.   Figure  Al  shows 

the  relationship  between  true  and  relative  velocities  and  angles.   In 

Figure  Al  the  vectors  are  "laid  off"  from  the  same  point.   The  actual 

situation  is  shown  in  Figure  A2 . 

Thus,  we  consider  a  coordinate  system  moving  with  the  observer 
(see  Figure  A3) . 
(or  Pursuer) '   The  y-axis  of  this  coordinate  system  is  coincident  with 

the  observer's  velocity  vectory.   P   is  a  fixed  point,  located  at  the 
origin.   The  point   E   (which  may  be  either  moving  or  stationary  in 
the  fixed  reference  frame)  moves  relative  to  P.   We  later  derive  the 
equations  of  motion  of   E  relative  to  P.   We  note  that  in  this  rela- 
tive coordinate  system,  there  are  two  factors  contributing  to  E's 
motion: 

(1)  rectilinear  motion  of  P  and  E, 

(2)  rotation  of  coordinate  system  when  P   turns. 
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(relative 
course) 


angle^/ 


:    -V 


Figure  Al.   True  and  Relative  Velocities  and  Angles. 
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Figure  A2.   Target's  Track  Relative  to  Observer. 
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Pursuer,  P 


w„ 
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Evader,  E 


<Jj  is  relative  heading  of  Evader 


0 


Figure  A3.   The  Relative  Coordinate  System. 


c.   The  Basic  Equations  in  a  Fixed  Reference  Frame. 

We  consider  the  motions  of  a  Pursuer,   P,   and  Evader,   E,   in 
a  stationary  coordinate  system.   We  consider  the  problem  of  the  Evader 
trying  to  break  contact  with  a  Pursuer  by  moving  out  of  the  Pursuer's 
detection  region,  which  is  circular  with  radius   d.   We  use  the 
notation  defined  in  section  Ilia,  of  the  main  text.   The  situation  is 
shown  below. 

y  k 


7 


_A:°>-_ 
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We  have  seen  above  that  the  non-zero  minimum  turning  radius,   R,   of 
the  Pursuer  yields  a  restriction  on  the  rate  of  change  of   a 


dct 

—  =  <j)s./R  where   -1  £  d>  £  1 

at     1 


Hence,  in  the  stationary  coordinate  system  the  surveillance-evasion 
problem  may  be  stated  as 


max  mm 
5'S1   6'S2  0 


dt  with   T   unspecified, 


dx, 


subject  to: 


dt 

Sl 

cos 

a, 

dya 

dt 

= 

Sl 

sin 

a, 

da 

=  (j)Sn/R  where   -1  i  4  i  1, 
dt     1 

dx2 

dF  =  S2  cos  B' 

dy2 

dT  =  s2  sln  6- 


where 


0  £  s   £  w 


0  £  s   £  w 


with  initial  conditions 


x1(t=0)  =  x°  ,  y1(t=0)  =  y°  ,  a(t-O)  =  aQ, 


x2(t=0)  =  X0  ,  y2(t=0)  =  y°, 


and  terminal  condition 


[xx(T)-  x2(T)]2  +  [yi(T)-  y2(T)]2  =  d 
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d.   The  Basic  Equations  In  the  Relative  Coordinate  System. 

We  now  transform  the  above  equations  to  the  relative  motion 
coordinate  system  discussed  previously.   In  this  new  system  x   is  the 
distance  the  Evader,   E,   is  from  P   in  a  direction  measured  perpendi- 
cular to   P's   heading.   In  this  relative  coordinate  system  there  are 
two  factors  leading  to  the  apparent  motion  of   E: 

(1)  rectilinear  motion  of   P  and   E,  and 

(2)  rotation  of  the  coordinate  system  when  P   turns. 

The  components  of  velocity  due  only  to  translation  are  given  by  (see 
Figure  A3) 


ij^)      "  s2  sin  ^' 


(-[£)   =  s2  cos  *   "  V 

since  P's  motion  is  always  directed  along  the  y-axis.   We  next  derive 

the  components  of  velocity  due  only  to  rotation  with  angular  velocity 

d8 
oj  =  -r~.      When   P   turns  to  his  right   (<t>  =  1)  ,   E   is  rotated  counter- 
dt 

clockwise  about  P.   We  take   w   to  be  positive  when  counter-clockwise. 
Then 

(^}  =  -wy  =  -(^)y  =  -s^/R, 
r 

f^)   =  oix  =  (||)x  =  s^/R. 
r 

Using  the  fact  that 

dt  ~  'dtj     ldtj  ' 
t       r 

equations  (1)  of  the  main  text  are  readily  obtained. 
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APPENDIX  B.   DESTROYER  TO  FIXED  DESTINATION 

In  this  appendix  we  derive  parts  of  the  solution  for  the  best  way  to 
steer  a  vehicle  with  a  minimum  turning  radius  to  a  fixed  destination  in 
least  time.   We  call  this  problem  "destroyer  to  fixed  destination."   It 
is  of  special  significance,  since  it  is  the  limiting  case  of  the  homicidal 
chauffeur  game  when  the  Evader's  speed  goes  to  zero.   Study  of  this  problem 
has  increased  our  insight  into  these  pursuit-evasion  problems.   The  new 
geometrical  construction  for  optimal  paths  was  first  suggested  in  our 
study  of  this  problem.   As  a  general  principle,  many  times  most  of  the 
significant  solution  aspects  of  a  differential  game  may  be  studied  by  con- 
sidering a  one-sided  version  of  the  problem. 

We  state  the  problem  and  then  present  the  details  of  analysis.   Next, 
we  discuss  our  new  geometrical  construction  for  optimal  trajectories  in 
such  problems.   Finally,  we  discuss  the  geometry  of  the  solution  and 
summarize  the  analysis  results.   Many  solution  steps  and  aspects  are  ex- 
plained in  elaborate  detail  in  this  appendix. 

a.   Statement  of  the  Problem. 

The  problem  is  to  determine  how  to  steer  a  constant  speed  vehicle 

with  minimum  turning  radius,   R  ,  from  any  point,   (x1  ,y1  )  ,  in  the  plane 

to  within  a  distance,  I    ,  of  a  terminal  point,  (x„,y9)  ,  in  the  least  time. 

In  a  stationary  coordinate  system  the  problem  is 

T 

dt   with  T   unspecified, 
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i  .  dx 

subject  to:     -r—     =  w  cos  8  , 
J  dt 

■77-  -     w  sin  6  , 
at 

—  =   cj)w/R  where   -1  <  d>  <  1  , 
at  —   — 

with  initial  conditions 

x(t  -  0)  =  xx  ,  y(t  -  0}  -  y1  ,   6(t  =  0)  =  e1  , 
and  terminal  condition 

[x(t  -  T)  -  x2]2  +  [y(t  =  T)  -  y2]2  =  £2 

We  transform  this  problem  to  the  relative  coordinate  system  of  Appendix 
A  in  the.  same  fashion  as  shown  there  to  obtain: 

min    dt   with   T  unspecified  , 

d>  Jo 


subject    to:  —     =     --  y<J>    , 


dx  w 

dt  R 


~r~     =     —  xd)   -  w     where     -1   <   d>   <   1    ,  (Bl) 

dt  R 


with  initial  conditions 


x(t  =  0)  =  xQ  ,    y(t  =  0)  »  yQ  , 


and  terminal  condition 


x2(T)  +  y2(T)  =  I2 
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b .   Development  of  Solution 
Hamiltonian,  H(t,  x,  p,  <$>) 

H(t,  j,  p,  ♦)  ■  1  +  p1(-|  y<J>)  +  p2(|  x*  -w)  ,  (B2) 

where  p.  =  t —   and  p„  =  — - —  are  dual  variables  and  J*  =  min    dt  . 
1   ax         r2    3y  J 


We  determine  the  extremal  control  from 

min  H(t,  x,  p,  <J>)  -*■  min{—  (f)(p0x  -  p..y}   subject  to   -1  sJ  ♦  i  L 

K     Z      J. 


Hence 


=  sgnCp^  -  p2x)  =  sgn  A(t)  ,  (B3) 


f+1   for   x  -  0 


where       A(t)  =  p,y  -  p?x  and   sgn  x  =  \ 

-1   for  x  <  0 


Boundary  Conditions  for  Dual  Variables 

Since  termination  is  any  point  on  a  curve,  we  must  have   p  =  (p  p  ) 
normal  to  this  curve  at   t  =  T(terminal  surface)   (see  [2]  p.  290).   We 
let   n  be  unit  normal  to  circle  (terminal  surface)  pointing  into  the 
state  space.   Then  n  =  sin  sit  cos  s  j  ,  and  we  have  p  =  an  .   Since 
the  constant  is  arbitrary,  we  let  it  be  1  and  hence 

p  (t  =  T)  =  sin  s  ,     p  (t  =  T)  =  cos  s  .  (B4) 

Usable  Part  of  Terminal  Surface 

Because  of  the  nature  of  the  relative  coordinate  system,  it  is  not 

possible  for  paths  from  the  state  space  to  end  anywhere  on  the  terminal 

2       2       2 
surface  x  (T)  +  y  (T)  =  I      .      Physically,  the  terminal  surface  is  a  circle 
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about  a  moving  point.   In  a  fixed  coordinate  system  this  circle  moves 
with  the  point.   Consider  a  line  perpendicular  to  the  point's  velocity 
vector  and  passing  through  the  point.   This  line  divides  the  circle  into 
two  parts:   one  part  is  ahead  of  the  point's  motion  and  the  other  is  be- 
hind.  The  problem  ends  when  a  fixed  point,  the  destination,  crosses  the 
moving  circle.   Clearly,  forward  motion  of  the  point  can  never  cause  a 
fixed  point  to  cross  the  rear  half  of  the  circle.   In  the  relative  coordi- 

-V 

-»-      • 

nate  system,  we  can  describe  this  condition  mathematically  as  n  •  X  <_  0  , 
where  n  is  unit  vector  normal  to  the  terminal  surface  and  pointing  into 
the  state  space  and 

,  dx  ' 

dT 
x  = 

dy_ 
■  dt 

is  the  vector  of  velocity  components  in  the  relative  coordinate  system  of 
the  moving  point  (destroyer).   That  part  of  the  terminal  surface  for  which 
"capture"  can  occur  is  called  the  useable  part  (see  p.  83  of  [30]). 
For  the  problem  at  hand  we  have 

n  =  sin  s  i  +  cos  s  j  , 

X  =  (--  y<J))i  +  (-  x<j>  -  w)j  , 

where  the  terminal  surface  is  parametrically  represented  by   s  as  shown 
below. 
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y 


/ 


X 


Hence,  we  have 


Then 


and  hence 


or 


x(t  =  T)  =  £  sin  s    and     y(t  =  T)  =  2.  cos  s 


X(t  =  T)  =  (-  ^  (J)  £  cos  s)i  +  (^  cj>  £  sin  s  -  w)j  , 
R  R 


n  •  X(t  =  T)  =  (-  —  ((>  £  cos  s)sin  s  +  (—  §    £  sin  s  -  w)cos  s 

R  R 


n  •  X(t  =  T)  =  -w  cos  s  ^  0   ,  (B5) 

defines  the  usable  part  of  the  terminal  surface.   Thus   cos  s  >_  0   for 
capture  to  occur  and  the  useable  part,   UP  ,  is  given  parametrically  by 

UP  =  {s|  -  \     <_  s  <_    \]    .  (B6) 

Equation  (B6)  simply  says  that  "capture"  only  occurs  in  the  "front  half" 
of  the  circle  about  the  moving  point. 

The  boundary  of  the  use£>le  part  of  the  terminal  surface,  BUP ,  divides 
the  terminal  surface  into  useable  and  non-useile  portions  (see  p.  83  [30]). 
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The  paths  which  terminate  at  the  BUP  are  then  the  barrier,  for  they 
separate  the  state  space  into  regions  from  which  paths  either  do  or  do 
not  "lead  directly"  to  the  UP.   We  have  noted  in  the  main  text  that  Isaacs' 
concept  of  the  barrier  corresponds  to  that  of  the  boundary  of  the  domain  of 
controllability  in  the  control  theory  literature  ([23]).   We  now  see  that 
we  can  determine  the  barrier  by  projecting  the  BUP  backwards  in  time  before 
capture. 

The  Adjoint  Equations  and  Their  Solution 
We  have  that 

dpl      3H 


=   '  ^    =   "Pi   D   *     > 


dt      3x     ^2  R 

dp2     3H 

Since  the  boundary  conditions  for  the  dual  variables  are  given  for   t  =  T  , 
it  is  convenient  to  let   t  =  T  -  t   and  integrate  the  adjoint  system  back- 
wards from  the  end.   Accordingly,  we  obtain 


dPi 

—  =  p2  -  $  Pi(t  =  0)  =  sin  s   , 


dp„ 

2        w  /  r\\ 

_-  -pll4,  p2(T  =  0)  =  COS  s 


We  may  combine  these  equations  as  follows: 

2  2 

d   Pt  dp.  0  dp  0 

rl  2      w    .  /WiK2  rl  /W(j>N2 


*  ■    -»i<ir    "     — 5    +  (T>  Pi 


,2  dt      R 

dx  dT 


=      0 
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Hence 


P1(t)    =  A   cos  —  t      +      B   sin  —  4>t 

with   initial   conditions 

p1  (t   =   0)    =  A  =   sin   s 

dpl 

—j-   (t    =   0)    =  —  4>B   =   p2(i    =   0)   —  4>    =  -  <J)    cos   s  ■*   B  =   cos    s    . 

Substituting   for    the   constants   and   simplifying  we    find 

Pi(t)    =   sin(s   +  |  <j>i)    .  (B7) 

Similarly,  it  may  be  shown  that 

P2(t)  =  cos(s  +  |  (J>t)  .  (B8) 

Solution  to  the  State  Equations 

In  the  "backwards  time"  we  have 

-^     =  —  (j>y  x(t  =  0)  =  £  sin  s  , 

-t^  =  -  f-  (jjx  +  w  Y(t  =  0)  =   kos  s  . 

dx       R 

We  combine  these  equations  to  determine  a  second  order  equation  for   x 

as  follows 

2 
d  x      w    dy       ,w  ,  v  2   .  w 

~2      =      R  *  dT   =   "(R  *>  X  +  R   *W   ' 
di 


or 


2  2 

d  x  ,w  ,  N  2      w 

-2  +  q  ♦)  x  =  — 

di 
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The  solution  to  the  above  equation  is  given  by 

/  \    A      w  ,  w       R 

x(t)  =  A  cos  —  (j)T  +  B  sin  —  <pi   +  —   , 
R  R       9 


where  the  constants,   A  and   B,  are  determined  by 

R  R 

x(t  =  0)  =  I   sin  s  =  A  +  —   ->  A  =  £  sin  s  —  — 


dx  ,     „^    w     ,     „v    w  w 

-j—  (t  =  0)  =  —  4>   y(i  =  0)  =  —  <J>  I   cos  s=-cf)B->B=£  cos  s  . 

dx  K  K  R 

Hence,  after  some  simplification,  we  obtain: 

x(t)  =  I   sin(s  +  £  <|>t)  +  y  (1  -  cos  -   91)  .  (B9) 

R        9  R 

Similarly,    it  may  be   shown   that 

v(t)    =    l   cos(s  +  ^  4>t)   +  —  sin  ^  <J>t  .  (BIO) 

R  (p  K 

Note  that  the  above  equations  (and  also  (B7)  and  (B8)  hold  from  the  terminal 
surface  (t  =  0)   until  a  transition  surface  is  reached. 
Transition  Surfaces  and  Termination  of  the  Barrier. 

We  have  seen  (equation  (B3))  how  the  "steering"  variable   <J>   (this 
variable  determines  the  rate  of  change  of  heading  for  the  moving  point  in 
the  fixed  coordinate  system)  is  determined  by 

9  =  sgn  A(t)   ,  (B3) 

where  A(t)  =  p,y  -  p9x  .   There  is  a  problem,  however,  at   t  =  T  ,  since 
A(t  =  T)  =  A(t  =  0)  =  sin  s(£  cos  s)  -  cos  s (£  sin  s)  =  0  .   We  can  overcome 
this  by  a  continuity  (with  respect  to  t)  argument.   Moreover,  we  have 
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dA  dpl  dy  dp2  dx 

dT     =     ~d7  y  +  Pl   dT     "     "17  X   "  P2   dT 


=    (p2   ^   <J))y  +  P1C-  ^  *x   +  w)    -    (-Pl  j<$    -   p2(|  y<j>) 


Hence 


—  =   pnw  =  w  sin (6   +  —  <J)t)      with     A(t    =   0)    =   0    , 
dx  1  K 

and    then  we   finally  obtain . 

R  w 

A(t)    =  —  {cos    s   -   cos(s   +  —  ((jt)}    .  (Bll) 

<p  R 

Thus,  for  0  <_   t  <_  t_  (where   x.   will  be  determined  presently), 

,  -1   for  -  —  <_   s  <  0 

♦  CO  =   ;  TT 

+1   for    0  <  s  <_  — 

Since  optimal  trajectories  are  symmetric  about  the  y-axis,  we  only  consider 
solution  behavior  in  the  right  half-plane  of  our  relative  coordinate  system, 

TT 

Consequently,  for  0  <  s  <_  -r-  ,  we  determine   t   as  follows:   it  is  the 
first  time  that  A(t)  =  0  after  t  -   0  .   This  happens  when 

w 

IV 


cos  s  =  cos  (s  4-  —x.)   , 


which  is  precisely  when 

o  w 

2tt   -   s   =   s  +  —  x      . 

R      i. 
Hence,    for     0   <    s   <_  tt/2    ,   we  ha>?=e 

T      =    2R    ^   -   s)    .  (B12) 

i.         w 

Equation  (B12)  determines   T-   such  that   4>(x)  =  1   for   0  <_   x  <^  x   .   For 
x  >  x1  ,  we  have  A(x)  <  0   (we  have  not  proved  this)  and  hence  <f>(x)  ~  _1 
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on  optimal  trajectories  in  some  interval  past   t   .   Hence,  the  locus  of 
such  points  where   (B12)   holds  defines  a  transition  surface. 

For   s  =  tt/2  ,  we  have  the  barrier  and  hence  <$>(i)    =1   on  this  "right 
barrier"  for  0  ^_  t  <_  Rtt/w   .   From  (B9)  and  (BIO),  the  equations  of  the 
barrier  are 


w 


x(t)  -  R  =  -(R  -  I)    cos  -  T   , 

K 


w 


yd)  =    (R  -   i)    sin  -  i 


which  we  may  write  as 

x(t)  -  R     '  co 
I   y(t) 


w 
S  R  T 


w 
-sin  -  t 


that 


w 
sxn-x 


w 
R 


I   -  R 


r  t  /     0 


This  equation  says  f.  to  determine  a  point  of  the  barrier  at  time   t  <_  —  tt  , 

w 
we  rotate  the  point  x=£-R,y  =  0   through  an  angle  —  t   in  the 

negative  (counter-clockwise)  sense  about  the  point   x  =  R  ,  y  =  0  .   Hence, 

for  0  <  T  <  —  tt  ,  the  barrier  traces  out  the  curve  shown  below  in  Figure  Bl 

—   —  w 


Barrier—' 


\ 


(-R,0) 


4 


I      (0,0) 


V 


V 


(R,0) 


Figure  Bl .   Barriers  to  Destroyer  to  Fixed  Destination  Problem. 
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Now  we  shall  explain  why  this  barrier  must  terminate  at   t   =  tt2/w. 
We  have  previously  discussed  this  subject  in  section  Illb  of  the  main  text 
(see  also  pp.  210-214  of  [30]).   To  summarize,  there  are  two  equivalent 
criteria  for  termination  of  the  barrier: 

(1)  (Isaacs)  due  to  a  change  in  the  orientation  (direction  of  travel) 
of  the  barrier  curve  this  semipermeable  surface  cannot  be 
extended, 

(2)  (Taylor)  the  barrier  terminates  if  its  extension  would  lead  to  a 
"multicovering"  of  extremals  in  the  region  of  extension  (with 
the  extension  being  non-optimal) . 

For  the  problem  at  hand,  for   t  <  t..  =  ttR/w  and   s  =  tt/2   there  would  be 
a  cusp  (tangent  to  curve  continuous  but  both  dx/di   and   dy/di   change 
sign),  since  the  extension  (we  have  not  proved  this)  would  be  an  arc  of  a 
circle  with  center   (-R,0)  with   <j>  =  -1  ,  i.e.,  curve  "changes  orientation 
and  there  can  be  no  semipermeable  continuation"  ([30]  p.  211).   We  recall 
that  for   x  =  x(x)   and  y  =  y(x) 

fdx\ 

dy     ,di  , 
dx     dx 

dx: 

Thus  we  can  have  the  tangent,   dy/dx  ,  continuous  but  both  dx/dy  and 
dy/dx   change  sign.   The  curve  has  a  cusp  at  such  a  point.   For  the  problem 
at  hand,  the  continuation  of  the  barrier  past   T-   would  produce  such  a 
cusp,  but  a  cusp  is  not  necessary  for  termination  (see  homicidal  chauffeur 
game  Appendix  C) . 
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We  might  also  argue  along  the  lines  of  (2)  above  that  if  (in  the 
game  of  degree)  the  barrier  were  to  be  extended,  it  would  intersect  ad- 
jacent extremals  causing  a  "multi-valued"  solution.   Hence,  we  discard  the 
dashed  portion  shown  in  Figure  Bl . 
Continuation  of  Solution  Past  Transition  Surface 

Continuation  past   t,  =  2R/w(tt  -  s)  for   0  <_  s  <  tt/2  ,   results  in 
an  arc  of  a  circle  with  center   (-R,0)   with  travel  for  increasing   t   in 
positive  (counter-clockwise)  direction,  i.e.,   <J>  =  -1  .   Time  has  not 
allowed  all  such  analysis  details  to  be  worked  out. 
The  Singular  Solution  (Universal  Surface) 

Fcr  the  problem  at  hand,  the  Hamiltonian  is  a  linear  function  of 
the  control  variable  <J>  , 

H(t,x,p,<|>)  =  <J)  |  (p2x  -  pxy)  +  (1  -  p2w)  (B14) 

with  the  control  being  determined  by  (B3)  except  for  when  the  coefficient 
of   d>   vanishes  for  a  finite  interval  of  time.   In  this  case  we  have  a 
singular  solution  [32],  [33]  for  which  the  necessary  condition  of  maximizing 
the  Hamiltonian  (with  respect  to  the  control  variable   4>  )  does  not  provide 
us  with  a  well-defined  expression  for  the  extremal  control.   Isaacs 
([30]  Chapter  7)  uses  the  terminology  universal  surface. 

A  singular  extremal  is  determined  from  the  conditions  [32],  [33] 


9H                 d   3H     n      d2  3H   n              a    a 

—  =  0     and     -j—  —  =   0  ,    — =■  —  =  0  ,  etc.  as  needed 

3d                 dt   dd>           ,  z  dcp 

dt 
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For  the  problem  at  hand  we  also  have  H(t,x,p,<}))  =  0  ,  since  the  termination 
time  is  unspecified.   Also,  above  equation  (Bll)  we  saw  that  dA/dx  =  p  w  . 
Thus,  since   8H/3<j>  =  -  —  A(t)  ,  we  have  the  following  equations  for  a 
singular  subarc 


1  -  p-w  =  0   , 


f(p2x  -  Ply)  =  0  , 

2 
and  p  w  /R  =  0  . 


Hence,  we  see  that  on  a  singular  subarc  we  have 


p2(t)  =  1/w  >  0  , 


Px(t)  -  0   , 
x(t)  =  0  . 


(B15) 


The  singular  control  is  determined  by  the  above  conditions  that  the  dual 
variables  are  constant  for  a  finite  interval  of  time.   Recalling  that 
dp ,/dt  =  -<|>p  w/R  and   dp„/dt  =  <j>p..w/R  ,  we  see  that   <j>(t)  -  0   is  the 
required  singular  control.   A  physical  interpretation  is  enlightening: 
once  the  destination  is  straight  ahead   (x(.t)  =  0)  ,  the  destroyer  steers 
a  straight  course   (*(t)  =  0)  . 

We  must  further  test  to  see  if  this  singular  solution  can  yield  the 
optimal  return;  i.e.,  minimum  time.   A  necessary  condition  for  a  singular 
subarc  to  yield  the  minimum  return  [34]  is  (see  also  [30]  pp.  187-188) 


£  1^5 (!<$  io- 
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2 
We  have  already  shown  that   d/dt  (9H/9cj>)  =  p  w  /R  .   Hence 

d2  fSHl    dpl   2,„  ,  _2 


dt' 


2   3*   =^F  W  /R  =  -*P2w(w/R) 


Thus 


3?{^2   (fj)   '  -P2«WR)2  (B16) 

But  on  the  singular  subarc  (B15)  must  hold,  so   p9(t)  =  1/w.   Hence,  on 


the  singular  surface 


Id!2   PIS 


k\£  \m  -  -^2  <  o . 


and  the  necessary  condition  is  met. 

c .   The  New  Geometrical  Construction  for  Optimal  Trajectories. 

The  following  is  given  to  show  our  original  motivation  in  developing 
this  geometric  interpretation  of  the  solution  to  the  state  equations. 
Others  may  prefer  the  analytic  geometry  of  transformations  of  ordered 
pairs,  but  we  usually  remember  such  things  by  considering  complex  numbers. 
We  start  by  recalling  ([11]  p. 8)  the  well-known  geometric  interpretation 
of  complex  numbers  in  which  a  complex  number  is  represented  by  a  point  in 
the  plane.   With  this  interpretation,  we  may  then  develop  that  multiplication 
by  a  complex  number  of  unit  modulus  corresponding  to  a  rotation.   We  now  look 
for  an  algebraic  representation.   It  is  well-known  that  the  field  of  com- 
plex numbers  is  isomorphic  to  a  field  of  2x2  matrices  with  the  correspon- 
dence being  given  by 

a       "b  ' 

a  +  lb  ->- 

b      a  /  . 
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Thus,  we  see  that  the  equation  (giving  the  locus  of  the  transition  surface, 
see  above) 

;x(x .)  -  R       /  cos  2s     -sin  2s    I   sin  s  -  R 

j  (£17) 

yd-,)  sin  2s      cos  2s /  \i   cos  s     /   , 

\     i    /  \ 

is  equivalent  to  (see  pp.  2-5  in  [11]) 

12s 
(xd-,)  -  R)  +  iy(i  )   =  e    [(&sin  s  -  R)  +  i£  cos  s]  , 

and  thus  represents  a  rotation  of  the  point   x  =  £sin  s  ,   y  =  kos  s 
through  an  angle   2s   in  the  positive  sense  (counter-clockwise)  about  the 
point  x  =  R  ,  y  =  0  .   After  we  recognize  this,  we  may,  of  course,  use 
analytic  geometry  to  reach  the  same  conclusion. 

Thus,  we  consider  optimal  trajectories  in  the  right  half-plane  for 
0  _<  x  <_  t   =  2R(tt  -  s)/w.   We  may  write  equation  (B9)  as 

x(t)  -  R  =  (i   sin  s  -  R)  cos  —  x  +  I   cos  s  sin  —  t  .   (B18) 

R  R 

Similarly,  equation  (BIO)  becomes 

y(i)  =  I   cos  s  cos  —  t  -  (£  sin  s  -  R)  sin  tt  t.       (B19) 

K  K 

We  may  write  the  above  as 

for   0  <_   s  <_  tt/2     and     0  <_  x  <_  x-  =  2R/w(tt  -  s) 


x(x)  -  R      i  cos  —  x       sin  —  x        x(T)  -  R 

R  R    ;    s 


(B20) 


where 


y(x)        \  -sin  ^  x       cos  ^  x  /        y(T) 


(T)  =  I   sin  s  ,   y(T)  =  I   cos  s. 
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The  geometric  interpretation  of  (B20)  is  that  the  point   x(t)  ,   y  (t) 
which  lies  on  a  trajectory  terminating  at   x(T),  y(T)  may  be  obtained  by 
rotating  the  point  x(T)  ,  y(T)  through  an  angle  wx/R   in  the  negative 
sense  about  the  point   x  =  R  ,  y  =  0  .   We  recognize  the  example  above 
(B17)  as  the  special  case  of  (B20)  when  x  =  x   =  2R(tt  -  s)/w. 

Let  us  see  how  the  above  geometric  interpretation  is  useful  in 
sketching  the  transition  surface.   From  (B12)  we  see  that  a  change  in 
steering  occurs  at  different  times  for  different  trajectories.   We  have 


s  =   0 

tt/4 
tt/2 


w  . 

R   Tl    =    2TI 


3/2TT 


By  considering  the  geometric  interpretation  of  (B20) ,  we  obtain  the 
picture  shown  below  in  which  the  transition  surface  is  a  dashed  line 


A 


Barrier -y 


Barrier 


(-R,0) 


'*  — Transition 
Surface 


> 
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d.   Geometry  of  the  Solution 

In  this  section  we  summarize  the  results  of  analysis  by  drawing  a 
picture  of  the  optimal  trajectories  with  the  control,   4>  ,  being  indicated 
in  various  regions.   The  singular  subarc,   x  =  0  ,  and  paths  leading  to 
it  (we  have  not  done  this  analysis)  are  also  given.   This  is  shown  in 
Figure  B2.   Again,  the  reason     this  "one-sided"  problem  has  been 
studied  in  such  detail  is  that  it  is  the  limiting  case  of  the  homicidal 
chauffeur  game  when  the  Evader  is  immobile,  i.e.,   w.  =  0  . 


)=-l 


Barrie 
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T 
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Figure   B2.      Optimal   Trajectories   for   Destroyer   to   Fixed  Destination 
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We  should  note  that  the  above  solution  is  in  the  relative  coordinate 
system.   We  discuss  briefly  how  the  optimal  paths  look  in  a  fixed  coordi- 
nate system.   Such  is  shown  below, where  the  point   P   is  the  movable  point 
and  three  destinations, 


B 


w 


A,  B,  and  C  are  shown.   These  points  correspond  to  A,  B,  and  C  of 
Figure  B2.   Point  A  is  reached  by  turning  as  sharply  as  possible,   <J>  =  1 
Point   B  is  reached  by  turning  sharply  and  then  a  straight-on  approach. 
The  last  part  is  the  singular  subarc  where   <j>  =  0  .   Point   C   is  within 
the  minimum  turning  radius,   R,  and  some  maneuvering  is  required  by   P  . 
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APPENDIX  C 
ORIGINAL  HOMICIDAL  CHAUFFEUR  GAME 

In  this  appendix  we  derive  parts  of  the  solution  to  Isaacs'  homicidal 
chauffeur  game  [30].  We  do  so  because  of  its  close  similarity  to  the  sur- 
veillance-evasion game  (see  section  III  of  main  text) . 

We  state  the  problem  and  present  details  of  its  solution.   Next,  we 
discuss  our  new  geometrical  construction  for  optimal  trajectories  in  this 
problem.   Finally,  we  discuss  the  geometry  of  the  solution  and  summarize 
the  analysis  results.   A  more  detailed  discussion  of  many  solution  steps 
is  to  be  found  in  Appendix  B. 

a.   Statement  of  the  Problem. 

The  problem  is  to  determine  how  a  Pursuer,  who  travels  at  a  constant 
speed  w, ,  with  a  minimum  turning  radius  R  ,  should  steer  to  capture 
in  minimum  time   T  an  Evader,  who  travels  at  constant  speed  w_  ,  has  no 
restriction  on  maneuverability,  and  tries  to  maximize  the  capture  time   T. 
Capture  conditions  are  defined  by  Pursuer  and  Evader  being  separated  by  a 
distance  I    .   In  a  stationary  coordinate  system  the  problem  is 

rT 


min   max 

4>     6 


dt  with  T  unspecified  , 
0 


subject  to:     dx, /dt  =  w..  cos  a  , 


dy. /dt  =  w1  sin  a  , 


da/dt  =  4>  W../R    where  -1  <_  <j>  <_  1  , 


dx„/dt  =  w„  cos  3 


dy2/dt  =  w   sin  $  , 
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with  initial  conditions 


X;L(t=0)  =  x1  ,   y1(t=0)  =  y1  ,   a(t=0)  =  aQ 

x2(t=0)  =  x2  ,   y2(t=0)  =  y2  , 

and  terminal  conditions 

[Xl(T)  -  x2(T)]2  +  [Yl(T)  -  y2(T)]2  =  I2    . 

We  transform  the  above  problem  to  the  relative  motion  coordinate  system  of 
Appendix  A  in  the  same  fashion  as  shown  there  to  obtain: 


mm  max 

4   i> 


T 

dt   with  T   unspecified  , 

0 


subject   to:  dx/dt   =  -w..  /R  y<}>   +  w»    sin  ^    , 


dy/dt   =  w../r   x<f>    -  w.    +  w     cos   ty        where      -1   <_<()<_  1    ,  (CI) 


with  initial  conditions 


x(t=0)  =  xQ  ,  yCt=0)  =  yQ  , 


and  terminal  condition 


x2(T)  +  y2(T)  =  £2   . 


b .   Development  of  Solution. 

Hamiltonian,  H(t  ,x,p  ;<j)  ,\ji) 

Wl 
H(t,x,p;<|>,i|0  =  1  +  pxC-  —  y  *  +  w2  sin  ^ 


Wl 

+  P2(  y"  X(^  ~  wi  +  w2  cos  ^ 


(C2) 
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whe 


re     p1 (t)    =   9J*/9x    (t)      and      P2CO    =   8J*/Sy    (t)      are  dual  variables 


and 


J*   =  min  max 

4>        i> 


dt   =  max  min 

0  \\>        <j> 


dt  .   We  determine  extremal  strategies 


from 


min  max  H(t,x,p;4>,^)  -> 

wi 

min  {-  —  4>(p1y  -  P2*)>  » 


and 


max  {w^Cp,  sin  i>  +   p~  cos  ty)} 


Hence 


=  sgn  A(t)  , 


(C3) 


where 


A(t)  =  pxy  -  p2x  . 


To  maximize  with  respect  to   l|)   it  suffices  to  consider 


f(ijj)  -  P-,  sin  ty   +  p   cos  i>    . 


A  necessary  condition  is  that 


and  hence 


df/d^t  =   0  =  p1    cos   \p   -  p„    sin  \\>    , 


tan  i>   =  P1/P2 


Thus,  to  maximize  we  must  have 


sin  ii   = 


/   2 


cos  ty    = 


/  2     2 
^Pi   +  Po 


(C4) 
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Boundary  Conditions  for  Dual  Variables 

We  must  have  p  =  (p   p  )   parallel  to  the  normal,   n  ,  pointing  into 
the  state  space  to  the  terminal  surface.   Now,  since 
n  =  sin  s  i  +  cos  s  j   ,  we  have 

p  (t  =  T)  =  sin  s   ,   p  (t  =  T)  =  cos  s 

X  w 

Usable  Part  of  Terminal  Surface 

We  have  that 

-»■-»■-*■ 
n  =  sin  s  i  +  cos  s  j   , 

->      wn  w. 

X  =  (-  —  y4>  +  w2  sin  40  i  +  Cy  xcf)  -  w  +  w  cos  i^)j 


where  the  terminal  surface  is  parametrically  represented  by   s  as  shown 
below. 


Hence,  we  have 


x(t  =  T)  =  I   sin  s   and   y(t  =  T)  =  £  cos  s 


Hence, 


w. 


n  *  X(t  =  T)  =  (- 


I   cos  s  +  w.  sin  ijj)sin  s 


w. 


+  (-—  tj)  I   sin  s  -  wn  +  w0  cos  ^)cos  s 
K  1     A 
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and  when  we  recall  that 


Pj/t  =  T) 

sin  <Kt  =  T)  =  ■  ■  ' =  sin  s  , 

/pl  +  P2 


P2(t  =  T) 

cos  i|)(t  =  T)  =     ■         =  cos  s  , 


pl  +  p2 

we  obtain 

■> 
n  •  X(t  =  T)  =  w  -  w  cos  s  <  0    ,  (C6) 

where  we  also  assume  w_  <  w1  (otherwise  capture  is  impossible  unless  the 
Evader  is  stupid).   Hence,  the  useable  part  of  the  terminal  surface  is 
given  by 

UP  =  {s|-S  <_  s  <_  S  where  cos  S  =  w  /w   ,   0  <_   S  <  tt/2}  .   (C7) 

The  Adjoint  Equations  and  Their  Solution 

We  have  that 

Wl 
dP;L/dt  =  -  3H/3x  =  -p2  —  (f) 


w 
dp2/dt  =  -  9H/8y  =  V±   —  (ft 

To  integrate  backwards  from  time  of  capture,   T  ,  we  let   t  =  T  -  t 

to  obtain 

Wl 
dp-^di  =  p2  —  4>  p  (t  =  0)  =  sin  s 

w 

dp2/di  =  -  p  —  4>  P2(t  =  0)  =  cos  s 
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We  combine  these  equations  to  obtain  a  second  order  differential  equation 
for  p.   as  follows 

.2   ,,  2   dp2  Wl        ,  Wl   ,2 


d  pi/dT  =^7  ¥"* =  ~pi(t  *} 


or 


2     2     Wl   2 
dZPl/dTZ  +  (  -f   *)^Pl   =   0 

The  solution  to  the  above  equation  is  given  by 

Wl  Wl 

p  (t)  =  A  cos  —  (J)T  +  B  sin  tt-  <£t   , 
IKK 

where  the  constants,   A  and   B  ,  are  determined  by 

p1  (t  =  0)  =  A  =  sin  s  , 

w  w  w 

dPl/dx(T   =   0)    =  —  <j>B  =   p2(x   =   0)  -^      =  -±  <j,   cos   s    , 

or  B  =  cos   s 

Hence,  after  some  simplification,  we  obtain: 

w 
P;l(t)  =  sin(s  +  "y  *t)  .  (C8) 

Similarly,  it  may  be  shown  that 

w 
P2(t)  =  cos(s  +  y-   <pi)  .  (C9) 

Taking  note  of  equation  (C4),  we  also  obtain 

Wl  Wl 

sin  \\>   =   sin(s  +  —  <}>t)        and        cos   ty  =   cos(s  +  —  4>t)    .  (CIO) 

K  K 
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Solution  to  the  State  Equations 

In  the  "backwards  time"  we  have 

Wl  Wl 

dx/di  =  —  (J)y  —  w  sin(s  +  -r-  <J>t)  x(t  =  0)  =  Jisin  s  , 

K         Z  R 

Wl  Wl 

dy/dx  =  -  -r-^x  +  w..  -  w0  cos(s  +  — -  (fit)        y  (r  =  0)  =   cos  s  . 
R        ±  Z  R 

We  combine  these  equations  to  determine  a  second  order  differential  equation 
for  x  as  follows 

2    2   Wl   d       Wl  Wl 

d  x/dr   =_^*~^""W2X*  cos(s  +  X  *T^ 

Wl   2    Wl  Wl 

=  -  ("jj-  *)  x  -  —  4){w2  cos(s  +  —  4>t)  -  w1> 

wl  wl 

-  W2  —  <J>  C0S(S  +  —  (j)T) 

or 

2    2    Wl   2      Wl  Wl       Wl  2 

d  x/di   +  (-j-   <j>)  x  =  -2y=-  <|,  w2  cos(s  +  X  *T)  +  R    *'      (C11) 


with 

x(t  =  0)  =  I   sin  s  , 

Wl 
dx/dt(T  =  0)  =  —  <j>  I   cos  s  -  w0  sin  s 
R  Z 

The  general  solution  to  the  above  inhomogeneous  differential  equation  is 

given  by 

x(t)  =  xu(t)  +  x  (t)  ,  (C12) 

where 
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x  (t)   is  the  general  solution  of  the  homogeneous  equation, 

x  (t)   is  any  particular  solution  to  the  inhomogeneous  equation. 

The  general  solution  of  the  homogeneous  equation  is  given  by 

w  w 

xh(t)  =  A  cos  —  <j)i  +  B  sin  —  <|>t  .  (C13) 

We  use  the  method  of  variation  of  parameters  ([20]  pp.  72-73)  to 
find  a  solution  to  the  inhomogeneous  equation.   (This  is  quite  messy  but 
I  have  not  found  an  easier  way.)   We  use  two  linearly  independent  solutions 
to  the  homogeneous  equation. 

Wl 

U   (f)  =  COS  —  <j>T   , 
Wl 

u2(t)  =  sin  —  4>t  , 

and  determine  v1  (t)   and  v^(x)   where  the  particular  solution  is  assumed 
to  be  of  the  form 

xp(x)  =  v1(t)u1(t)  +  v2(t)u2(t)  .  (C14) 


Hence 


dx    dv-         du    dv  du 

■  u,  +  v,  -r^  +  -~  u„  +  v. 


dx     dx    1    1  dx     dx    2    2  dx 

Now  we  set 

dv       dv 

— Au   +— 2U    e   0  ,  (C15) 

dx   1    dr   2 

leaving 

dx         du        du_ 
p  1         2 

"d7  =  vi  "dT  +v2dT' 
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so  that 


2     2 
d  x  /dx 
P 


2 
dv   du       d  u    dv   du 

-  v,   -, ■  v  +  — r-  — —  +  V2 


dx    dx 


dv   du 
dx    dx 


ri  ~7^ 

dv„   du 


dx    dx 


w. 


,2 

d  u, 


dx" 


+ 


t    ~£~   CT*)2(vlul  +  V2U2}  ■ 


since  u. (x)   and  u«(x)   are  solutions  to  the  homogeneous  equation. 
Hence,  using  (C14) , 


d2x 


dx 


w..    0     dv.   du.    dv.   du. 
2    ^R  vy   p    dx    dx    dx    dx      v  '  ' 


(C16) 


where      R(x)      is    the   right   hand   side   of    (Cll)    and 


w  WW 

R(x)    =   -2-~  4>  w2   cos(s  +  —■  cj)x)   +  -^ 


(C17) 


To  summarize,  (C15)  and  (C16)  give  us  two  equations  for   dv  /dx   and  dv  /dx 


du   dv      du0   dv 

+  -A     -A     =      R(x) 


dx    dx 


dx   dx 


dv. 


dv, 


+  u. 


1   dx    2   dx 


=  0 


Solving  these  equations  by  Cramer's  rule,  we  find  that 

du. 


dv. 


j  R(t) 
0 


dT 


dx 


du 


1 

dx 


dx 


R(t)u2(t) 

'   W1 


(C18) 
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Similarly, 

tl  =   R(T)U1(I)  (C19) 

dT     w 

Substituting  (C17)  and  the  definitions  of   u   and  u   into  the  above, 
we  find  that 

dv-  w  w 

—j—  =  w0  sin(s  +  2  —  (j)t)  -  w„  sin  s  -  wn  sin  -r-  At  , 
aj  Z  R         2  1      R 

and 

dv  w  w 

—-5—  =  -w0  cos  s  -  w0  cos(s  +  2  —  <p"r)  +  w  cos  —  <j)T  . 

Q.T        Z  Z  R  i       R 

Integration  of  these  equations  yields 

W2R  Wl  R     Wl 

v,  (t)  =  -  ■= cos(s  +  2  —  4>t)  -  tw„  sin  s  +  —  cos  —  <pi  , 

1  ZW..  <p  R  Z  <p       R 

(C20) 
and 

W2R  Wl  R  Wl 

v0  (t)    =  -  tw_   cos   s   -  -z — -     sin(s  +  2  —  <px)   +  —     sin  —  4>t. 
z  z  zw-  <p  R  <j>  R 

(C21) 

Substituting  (C20)  and  (C21)  and  the  definitions  of   u   and   u„   into 
(C14),  we  obtain  after  some  manipulation 

W2R  Wl  Wl       R 

x  ^T)  =  ~  o — 7  cos(s  +  —  <pr)  -  tw   sin(s  +  —  (pi)  +  -         (C22) 
p         zwn  (p  R  z  R        <p 


Combining  (C13)  and  (C22) ,  we  see  that  the  general  solution  to  (Cll)  is 
given  by 
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w  w        w  R  w 

x(t)  =  A  cos  —  4>t  +  B  sin  -r-  <j>x  -  -z — r  R  cos(s  +  -t—  (pi) 

K  K        ZW1  <p  K 

W1        R 
-  tw.  sin(s  +  -~4>t)  +x   •  CC23) 

2  R         9 

The  initial  conditions  to  (Cll)  lead  to  the  following  determination  of 
the  constants  in  (C23) : 

A  =  Esin  s  +  75 — -  cos  s  —  T"    , 
2w1  (j>         <p 

w.R 

B  =  5-cos  s  -  ■= r  sin  s  •  (C24) 

2w14> 

Substituting  (C24)  into  (C23) ,  we  obtain  after  some  manipulation, 

w  w 

x(t)  =    (I  -   iw0)sin(s  +  ~  (j)i)  +  y  (1  -  cos  -i  (J)t).  (C25) 

Z  R        <p  R 

Similarly,  we  may  also  obtain 

w  w 

1       R      1 

v(t)  =    (I   -   twJcos(s  +  —  4>t)  +  —  sin  —  <px.  (C26) 

Z  K         <p       R 

Note  that  the  above  equations  (and  also  (C8) ,  (C9) ,  and  (CIO) )  hold  from 
the  terminal  surface   (t  =  0)   until  a  transition  surface  is  reached  or 
the  trajectory  terminates. 
Equation  of  the  Barrier 

We  have  seen  (see  Appendix  B)  that  the  barrier  (boundary  of  the  domain 
of  controllability)  is  an  optimal  trajectory  which  terminates  at  the  BUP. 
We  have  seen  in  (C7)  above  that  the  boundary  of  the  use&le  part  (BUP)  is 
given  by  s  =  +  S  ,  where  0  <_   S  <  tt/2  and 


cos  S  =  w„/w1  .  (C27) 
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Because  of  the  symmetry  about  the  y-axis  in  this  problem  it  suffices  to 
consider  the  right  barrier,  i.e.,  the  one  for  which  s  =  S  .   In  the  next 
section  we  show  that   <J>(t)  =  1   for   s  =  S  and   0  <_  t  <_  t.   where   t.   is 
also  determined.   Thus,  the  equation  of  the  right  barrier  is  given  by 

for   0  <_  t  <_  t 

w  w 

x(x)  =  U  -  iw2)sin(S  +  ~  t)  +  R(l  -  cos  j  t  )  ,  (C28) 

and 

w  w 

yd)  =  U  -  tw2)cos(S  +  —  t)  +  R  sin  — -  t  .  (C29) 

It  is  not  obvious  (as  Isaacs  and  others  seem  to  infer)  that  (C28)  and 
(C29)  are  the  equations  for  the  involute  to  a  circle.   It,  therefore, 
seems  appropriate  to  digress  and  review  some  analytic  geometry.   Our  dis- 
cussion follows   R.  Courant  ([12]  pp.  280-283  and  pp.  307-310). 

Consider  a  curve  represented  parametrically  by  x  =  x(t)  ,   y  =  y(t) 
in  the  plane.   The  curvature,   <  ,  of  this  curve  at  a  point  is  given  by 
k  =  da/ds  ,  where   tan  a  =  y/x  =  (dy/dt)/(dx/dt)  .   The  radius  of  curvature 
is  defined  by  p  =  1/k  .   For  a  given  point  on  the  curve,  there  is  a 
circle  of  curvature  corresponding  to  the  point.   This  circle  touches  the 
curve  at  the  point  and  there  has  the  same  sense  of  description  and  the  same 
curvature  as  the  curve.   Its  center  is  called  the  center  of  curvature.   Con- 
sider the  diagram  below.   At  any  point   (x,y)  ,  the  center  of  curvature, 
(S,n)  ,  is  given  by 
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tun) 


5  =  x  -  p  sin  a   , 
n.  =  y  +  p  cos  a   , 

where 

y 

sin  a  =  -  — ,u- 


(x,y 


/x   +  y 


2 


The  locus  of  such  centers  of  curvature  to  a  curve  is  called  the  evolute 
of  the  curve,   C  .   We  further  call  C   the  involute  of  its  evolute.   The 
evolute  is  the  "envelope"  of  the  normals  to   C  .   An  important  fact  that 
we  shall  use  later  is  that  the  tangent  to  the  evolute  of  a  curve  is  normal 
to  the  curve,  i.e.,   £  x  +  n  y  =  0 

If  we  have  a  curve   £  =  £ (a)  ,   n  =  n(o)   where  a     is  parameter,  then 
the  equations  of  the  involute  to  this  curve  are  given  by 


x  =  £  +  (a  -  aH   , 
y  =  n  +  (a  -  a)n   • 
For  a  circle,  represented  parametrically  by 

E,   =  -cos  t  , 
n  =  sin  t  , 
the  involute  is  given  by 

x  =  -cos  t  -  t  sin  t, 


(C30) 


(C31) 


y  =  sin  t  -  t  cos  t  . 

It  is  worth  noting  that  all  the  normals  to  the  curve  given  by  (C31)  are 
tangent  to  the  circle  (C30) .   We  show  the  curves  below.   Another  geometric 
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property  of  involutes  that  we  shall  use  is  that:   let, 


£..  =  distance  from  A   to   B  , 


£„  =  arc  length  from   B   to  D 


J>   =   distance  from   C   to   D, 


where 


AB  and   CD  are  normals  to  involute. 


Then 


£1  +  l2        l3 


(C32) 


To  show  that  (C28)  and  (C29)  are  the  involute  to  a  circle,  we  must 
show  that  they  are  of  the  form  (C31) .   To  do  this  we  consider 


w  w 

R  cos  —^  t  =  R  cos  (—  t  +  S  -  S) 
R  K 


w  w 

-  R  cos  (—  x  +  S)cos  S  +  R  sin(—  x  +  S)  sin  S 
R  R 


Considering  (C27),  we  see  that 


w., 


W2     Wl 


w 


R  /  2     2 

R  cos  -    :  =  R  — -  cos(v  t  +  S)  +  —  /w-   -  w    sin(-=p  t  +  S) 


B 


w,      R 


wi 


Thus,  (C28)  becomes 


w„      w 


x(t)  ■■  R  =  -  (R  —  )cos(-f-  t  +  S)  +  {[«,  -  7^  A/,2  -  w,2 


w 


W2  x  Wl 


w1    1 


w 


wl 

sin  (—  t  +  S) 


(C33) 


Similarly, 
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W  W  r- ■■■■  x  W      W 

yCx)  =  (R~)  sin(|i  +  S)  +  {U-^/  w.2  -  w  /  ]  -  (R  ~~-   )  -±   t} 


W„ 


COS  (—  T  +  S) 


(C3A) 


Considering  (C30)  and  (C31) ,  it  is  clear  that  (C33)  and  (C3A)  are  the 
equations  of  an  involute  to  a  circle  of  radius  RCw^/w..  )   and  with  center 
x  =  R  ,   y  =  0   .   The  equations  of  this  circle  are 

W2        Wl 
£  -  R  =  -  (R  —  )  cos  C-^-  t  +  S)  , 

W-  K 

W  W 

n .  =  (R  -f-   )  sin  (~-     t  +  S)  , 

W.  K 

where  the  involute  is  unwound  from 


Zi    .i, 

R  T    w2R  L     w 


-  /wn   -  w_   J  <  0 


X 


\ 


(R,0) 


v 


. 


/ 
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To  insure  that  the  involute  is.  unwound  under  all  circumstances,  we  must 
show  that  the  quantity  in  brackets  above  is  always  negative.   This  follows 
from  elementary  geometry  considerations  in  the  above  figure. 

2      W2   2    2 
JT  +  (R  —  )   <  R 

Wl 


or 


1 

Transition  Surfaces 
We  recall  that 


R  J     2      2 

I  -   —  rw,   -  w„   <  0   for  all  values  , 
w,    1     2 


<|>(t)  =  sgn  A(t)  (C3) 


where  A(t)  =  p,y  -  p„x  . 


Recalling  that 

x(t  =  0)  =  I   sin  s  ,   y(i  =  0)  =  I   cos  s, 
and  p1(t  =  0)  =  sin  s   ,   p2(t  =  0)  =  cos  s   , 

we  see  that 

A(t  -  0)  -  0  (C35) 


Also, 

dA/dx  =  77  y  +  px  S  "  "d?  x  '  p2  dT 

into  which  we  substitute  the  state  and  adjoint  equations  to  obtain 


dPl  dv  dp2  dx 

—  v  +  p,    , r~  x  -  p„  -t— 


dA/dx      =     p.w.    =  w.    sin(s  +  —  <£t)      with     A(t   =   0)    =   0 

1  1      i  K 

Integration  of  the  above  yields 


w. 


A(x)  =  t  ^cos  s  -  cos^s  +  ^  *x)  }  •  ('C36) 

(p  K 
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Thu 


s,  for   0  <_  t  £  x-   (where  x   will  be  determined  presently), 

-1  for  -S  £  s  <  0 
♦  CO  -    <  (C37) 

"  +1  for   0  <  s  ^  S  , 

where   S   is  determined  by  (C27).   By  the  symmetry  of  the  problem,  we 
concentrate  on  the  right  half— plane.   Consequently,  for   0  <  s  <  S   ,  we 
determine   t1   as  follows:   it  is  the  first  time  that   A(t)  =  0  after 


t  =  0  .   This  happens  when 


Wl 

COS  S  =  COs(s  +  -—  T.J 
R   1 

which  is  precisely  when 

Wl 

2  TT  -  S  =  S  +  -—  T1   . 
R   1 

Hence,    for      0   <    s   <_  S   <    tt/2    ,    we  have 

t,    =  —   (tt  -   s)    .  (C38) 

W- 

Equation  (C38)  determines   x,   such  that   <f>(x)  =  1   for   0  <  t  <_  t   . 

For   x  >  x   ,  we  show  later  that   A(t)  <  0  and  hence  <J>(x)  =  -1.   If 

trajectories  do  not  terminate  before  condition  (C38)  holds  then  the  latter 

gives  a  transition  surface. 

For   s  =  S  ,  where  cos  S  =  w  /w   ,  we  have  the  barrier  and  hence 

<|>(t)  =  1   on  this  "right  barrier"  for   0  <_  x  <^  2R/w  (tt  -  S)  .   From  (C28) 

and  (C29),  we  may  write  the  equations  of  the  barrier 

Wl  Wl 

x(t)  -  R  =  (H  -   twJcos  S  sin  —  t  +  {  (£  -  iw„)sin  S  -  R}  cos  —  t, 

Z  R  z  R 

Wl  Wl 

y(x)  =   -{  (H  -   iw0)sin  S  -  R}  sin  —  x  +  (£,  -  xw„)cos  S  cos  —  x, 
Z  R  Z  R 


which  we  may  write  as 
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/  x(x)  -  r\ 


Wl  Wl 

cos  — r-  x     sin  —-  t  \  (£  -  xw  )sin  S  -  R 

K  R  Z 


I 


\    yCO 


.    wi 

-sxn  —  x 


w„ 


cos 


T      \(£  -"  twJ  cos  S 


(C39) 


The  geometric  interpretation  of  (C39)  is  as  follows.   To  determine  a  point 
on  the  barrier  at  time   x  <_  (2R/w  )  (tt  -  S)  ,  we  start  at  the  point  x  =  £sin  S, 
y  =  Icos   S  on  the  terminal  surface  (this  is  the  BUP)  and  move  to  a  point 
which  is  a  distance   tw«   along  the  straight  line  connecting  the  first  point 
to  the  origin  (  and  toward  the  origin) .   We  now  rotate  this  latter  point 
through  an  angle   (w  /R)t   in  the  negative  (clockwise)  sense  about  the 
point  x  =  R  ,   y  =  0  .   Hence   for   0  <_   (w  /R)t  <_  2tt  -  2S  ,  the  barrier 
traces  out  the  curve  shown  below  in  Figure  CI. 


Figure  CI.   Barrier  in  Homicidal  Chauffeur  Game 
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Continuation  of  Solution  Past  Transition  Surface  and  Termination  of  Barrier 

Later  we  show  that  paths  terminating  on  terminal  surface  for  0  <_  s  <_   S 
may  all  converge  to  the  same  point  and  hence  terminate  there  (except  for  the 
barrier).   The  time  of  this  conveyence  depends  on  the  speed  of  the  Evader, 
w?  .   We  recall  that  in  the  limit  as  w„->-0   the  solution  approaches  that 
shown  in  Figure  B2.   If  w?   is  small  enough,  the  transition  surface  beyond 
the  termination  of  the  barrier  may  be  reached  by  a  trajectory  before  the 
trajectory  terminates  by  reaching  the  "focal  point."  Hence,  we  consider 
extension  of  the  barrier  which  we  know  continues  until   x..  .   Details  for 
other  trajectories  (if  they  exist  long  enough)  are  similar. 

Thus,  we  consider  the  continuation  of  the  right  barrier  past 
(w  /R)t   -  2  (tt  -  S)  .   From  (C8)  and  (C9),  we  have 

P1(t1)  =  -sin  S  ,      P2(t1)  =  cos  s  •  (c4°) 

From  (C25)  and  (C26)  ,  we  obtain 


x(t  )  =  (£  -  x  w  )(-sin  S)  +  R(l  -  cos  2S) , 


y(x  )  =  (£  -  x  w  )cos  S  -  R  sin  2S 


(C41) 


Later,  we  show  that  cf>(T)  =  -1   for   x  >  t1  .   We  assume  this  for  now.   The 
adjoint  equations  for   t  >  x    are 


P1(t1)  =  -sin  S   , 


d»l 

dx 

wl 

=   "P2T 

dp2 

Wl 

dx 

=    pi  T 

p-(x  )  =  cos  S 
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The   solution   to    the   above  equations    is      (for      x    >    x1    ) 


w 


PxCt)    =   -sin(S  +  ~   Ct   -  t    ))    =   sin  i>      , 


Wl 
P2(t)    =      cos(S  +  —  Ct   -  t^))    =   cos   ^ 


(C42) 


The  state   equations   for      t   >    t1      are 


Wl  Wl 

dx/dx   =  -  —  y  +  w^   sin(S  +  —   (t    -   t1))    , 


Wl  Wl 

dy/dx    =        —  x  +  w1   -  w2   cos(S   +  —   (t    -   t    ))    , 


(C43) 


with  initial  conditions  given  by  (C41).   A  rather  laborious  computation 

(we  omit  the  details)  yields  the  solution  to  the  above  as  (for   x  >  x..  ) 

w 
x(t)  =  -  a  -  iw2)sin(S  +  ~  (t  -  x^) 

w  w 

+  R{2  sin  -j-   (x  -  tx)  -  1  -  cos(2S  +  -^  (t  -  i±))}      , 


and 


Wl 


y(x)  =    (£  -  xw2)cos(S  +  —  (t  -  x^) 

w  w 

+  R{2  sin  -±   (t  -  t1)  -  sin(2S  +  -±   (t  -  x^)}   .       (C44) 

We  may  write  equations  (C44)  as  (for   x  >  x.) 

W1  W1         \   ' 

J    x(t)  +  R',    cos  y"  Ct  -  x1)    -sin  ~   (x  -  i ±)  \    ■(£  -  tw2)  (-sinS)+2R-Rcos2S 


\      y(x)      /      \sin  —  Ct  -  t±)  cos  ~-   (x   -  i^j  \U   -  xw2)cosS-Rsin2S 


(C45) 
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or   using    (C41) 

Wl  Wl              \ 

/x(t)    +  R\           cos  —  Ct-t  )         -sin—  Ct-t1)\     xCt^+w^t-t^cos  (=■  -S)    +R 

;      i  SI                  ] 

w1  w1              ■'  '                                                   i 


\  I 

\  y(i)      J         \sin  ^lCt-t1)  cos  Y*-1-1^/    V    yCx^l-WgCi— r^iainC^'S) 


(C46) 
The  most  important  aspects  of  the  geometric  interpretation  of  (C46)  are 

that  rotation  is  in  the  positive  sense  about  the  point   x  =  -R  ,  y  =  0 

and  the  involute  continues  to  unwind.   That  (C44)  does  indeed  trace  out 

an  involute  may  be  seen  by  writing  it  in  the  equivalent  form 

w         w 

x(t)  +  R  =  R  —  cos(S  +  —  (t  -  tJ) 
W.  R        i 


w„     w,  w. 


+  [{3R/l  -  (w9/w.)2  -  Vi   +   (R  —  )(t  -^  )]sin(S  +  ~   (t  -  t.))  , 

Z   1  W-.        K  K  1 

/> WW  w 

.  1  -  (w2/Wl)2  -  £}  -  (R  ^  )(t  -±   )]cos(S  +  X  (T  ~  Tl}) 


w  w 

+  R^  sin(S  +  -^  (t  -  xA)      ,  (C47) 

W-.  K  1 

which  is  an  involute  to  a  circle  of  radius   R(w~/w  )   and  with  center 
x  =  -R  ,  y  =  0  .   The  involute  is  unwound  from 


w,      w. 


-^  t)(R-  ■  )  -  -3RA  -  (w„/Wl)2  +  I   <  0 
R       w  2   1 


If  we  were  to  try  to  extend  the  barrier  given  by  (C44)  (or  any  equivalent 
form),  we  would  find  that  the  barrier  "bends  back  on  itself"  as  shown  by 
the  dashed  line  in  Figure  CI.   Hence,  by  the  arguments  given  in  Appendix  B, 
the  barrier  must  terminate  at   t,  =  (2R/w.. )  (it  -  S) 
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We  also  note  that  if  the  barrier  were  extended  for   t  >  t   ,  then 
the  tangent  to  this  curve  would  be  discontinuous.   We  denote 


dx 
dt 


+   as  being  the  limit  as   t^t   and   x  >  T   .   Similarly  for 

11^  _l  —   1 


dt  T  =  T. 


We  may  show  that  for  the  barrier  we  have 


dx 
dt 


T=T. 


=  w  {sin  S  -  (I   -  t  w  )/R}cos  S  , 


dy_ 
dt 


-  =  w  {sin  S  -  (£  -  T1w.)/R}sin  S   , 


and  hence  (as  we  knew  before) 


dy_ 
dx 


T  =  T. 


tan  S 


Also 


dx 
dt 


+  =   -w  {3  sin  S  -  (£  -  t  w  )/R}cos  S 


and 


dv. 
dt 


T=T. 


_  dy_ 
dt 


T  =  T- 


Hence  if   Tiw?   >  &      (causing  "focal  point"  in  field  of  trajectories,  i.e., 

1  l     the 
all  paths  exceptibarrier  terminate  before   t  =  t   ),  both  dx/dt   and 

"  The 

dy/dt   change  sign  at   i  =  t   .  ^curve  would  "almost  have  cusp"  at   i..   if  the 

barrier  were  extended  (except  for   dy/dx  being  discontinuous). 
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The  Singular  Solution  (Universal  Surface) 

Since  the  Hamiltonian  is  a  linear  function  of  the  Pursuer's  control 

variable  <f>  ,  the  maximum  principle  does  not  determine  the  control  when 

the  coefficient  of   <J>  vanishes  for  a  finite  interval  of  time  (see  Appendix 

B) .   Part  of  a  trajectory  for  which  this  occurs  is  called  a  singular  subarc, 

The  Hamiltonian  is 

Wl 
H(t,x,p;<f),^)  =  <j>—  (p2x  -  p^y)  +  w2(p1  sin  ^  +  P2  cos  ^)  +  1  -  P^ 

or  using  (C4),  we  have 

wl  T~l 2 

H(t,x,p;cj),i|;)  =  <f>—  (p2x  -  p  y)  4-  W^/p.^  +   p2    +   1  -  p2w   .  (C48) 

We  determine  the  conditions  for  a  singular  subarc  from 

H  =  3H/3<j>  =  (d/dt)(3H/3<|>)  =  0  (C49) 


Recalling  that  above  equation  (C36)  we  had  dA/dt  =  _w1p1   and  noting  that 
3 H/ 3tf)  =  (-w  /R)(A(t))  ,  we  have  from  (C49)  the  following  conditions  for 
the  singular  surface 


and  hence 


7    2  2 

W2  Pl  +  P2   +  1   ~   P2wi  =  °  ' 


Wl 

-g-  (p2x  ~  P]_y)  =  °    ' 


2 
w 

Pl^ "  "   °   • 


Pl(t)  =  0  , 

p0(t)  = >  0  ,  (C50) 

2  w1  -  w2 

x(t)  =  0   . 
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The  singular  control  required  to  yield  the  above  is  readily  seen  to  be 

7  2     2 
<J>(t)  =  0  .   Hence  recalling  (C4),  i.e.,  sin  i>   =  p^CO/Zp   +  p     , 

we  see  that  i>  (t)    =  0   on  the  singular  subarc .   We  have  not  traced  paths 
which  lead  to  x(t)  =  0  backwards  as  Isaacs  has  done  (.[30]  pp.  193  -  194). 
The  necessary  condition  for  optimality  [34]  on  the  singular  subarc  is  also 
met  since 

dt 
Determination  of  Capture  Criterion 

We  have  discussed  in  section  Illb  of  the  main  text  that  a  very  impor- 
tant question  is  whether  or  not  the  barrier  divides  the  state  space  into 
two  parts.   For  the  problem  at  hand,  if  the  barrier  does  not  divide  the 
state  space  into  two  parts,  then  (it  appears  as  though)  capture  can  occur 
from  any  initial  point,  i.e.,  the  entire  state  space  is  the  capture  zone. 
The  only  way  that  the  barrier  can  divide  the  state  space  into  two  parts  is 
for  the  "left"  and  "right"  barriers  to  meet  in  the  y-axis.   We  now  develop 
the  condition  for  this  to  occur. 

We  consider  Figure  C2  and  recall  the  relationship  between  two  normals 
to  the  involute  of  a  circle  given  by  (C32) .   When  capture  can  be  avoided, 
the  barrier  intersects  the  y-axis.   In  Figure  C2,  we  have 

I      =  Z2  +   £3  ,  (C51) 

i.e.  the  difference  in  the  length  of  the  normals  is  equal  to  the  distance 
on  the  perimeter  of  the  circle  (evolute)  between  points  of  tangency. 

For  capture  to  always  occur,  i.e.,  barrier  does  not  intersect  or  touch 
y-axis,  we  must  have 

R  >  SL      .  (C52) 


-I 
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-' 


Figure  C2.   Determination  of  Capture  Criterion. 
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We  also  have 


or 


and 


sin  S  =  £4/R  =  (&  +  O/R   , 


l0   =   R  sin  S   -   I      ,  (C53) 


£3  =  R(w2/w1)(  |  -  S)  .  (C54) 
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Now,  since   (y  -  S)  =  ?.^w  /  (Rw  )  ,we  have  (also  using  (C27)) 


sin(£~w  /(Rw  ))  =  cos  S  =  w./w   , 

and  hence 

l3   =  R(w2/wL)  sin"  (w2/w  )  (C55) 

Combining  (C51),  (C52),  (C53)  ,  and  (C55),  we  obtain  the  condition  for  the 
entire  state  space  to  be  the  capture  zone 


I   >  R{/l  -  (w2/w1)2  +   (w2/w1)  sin"1(w2/w1)  -  1}      (C56) 

c.   The  New  Geometrical  Construction  for  Optimal  Trajectories. 

We  consider  optimal  trajectories  in  the  right  half-plane  for 

0  <_  t  <_  t,  =  2R/w..  (tt  -  s)  .   Background  material  is  to  be  found  in  Appendix 

B.   For   0  <  t  <_  t   ,  we  have  that   <J>(t)  =   1  for   0  <  s  <^  S  ,  and  (C25) 

and  (C26)  may  be  written  as 

Wl  Wl 

x(t)  -  R  =  (i   -   tw  )cos  s  sin  -r-  t  +  {  (£  -  tw  )sin  s  -  R}cos  —  t   , 

Z  i\  Z  K 

and 

Wl  Wl 

y(x)  =  -{(£-  Tw„)sin  s  -  R}sin  —  t  +  (£-twJcos  s  cos  —  t  ,       (C57) 
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which  we  may  write  as 

for  0  <    s  <  S  where  cos  S  =  w_/w   and   0  <_  t  <_  t 


:Cl)    -  R 


y(t) 


wi 

COS   -r—   T 
K 


w 


i     \ 


sxnT  ' 


(X  -   tw  )sin  s   -  R 


Wl  Wl        J 

I -sin  —~-  x        cos  —  t   /    I      (£   -  twJcos   s  / 


CC58) 


The  geometric  interpretation  of  (C58)  is  as  follows.   To  determine  a  point 
on  an  optimal  trajectory  at  time  x   <_  min(jZ,/w_,  2R/w1  (tt  -  s))   (we  presently 
shall  show  why  addition  restriction;  see  above  near  (C39)  for  case  of  barrier), 
we  start  at  the  point  x  =  I   sin  s  ,  y  =  I   cos  s  where  s  e  UP  and  move 
to  a  second  point  which  is  a  distance   tw?   towards  the  origin  along  the 
straight  line  connecting  the  first  point  to  the  origin.   We  now  rotate  this 
second  point  through  an  angle   (w./R)t   in  the  negative  (clockwise)  sense 
about  the  point   x  =  R  ,  y  =  0  . 

We  have  used  the  above  geometric  interpretation  (C58)  to  discover  in 
the  homicidal  chauffeur  game  a  central  field  of  extremals  through  a  point 
on  the  barrier  corresponding  to   t  =  £/w_.   All  "primary  solution"  extremals 
(see  [30]  p.  278)  pass  through  this  same  point.   For   0  <_   x  <_  £/w„  ,  we 
suspect  that  on  any  primary  trajectory  there  is  no  point  conjugate  to  x   =  0, 
but  we  cannot  check  this  by  the  Jacobi  condition,  since  the  strengthened 
Legendre-Clebsch  condition  is  not  satisfied,  i.e.,  H   =0   (see  [9]  p.  181, 
also  [44]  p.  398).   The  significance  of  a  conjugate  point  is  that  the 
primary  solution  terminates  at  this  locus  of  focal  points  to  the  UP  of  the 
terminal  surface. 

We  can,  however,  investigate  the  existence  of  conjugate  points  by  the 
use  of  our  geometrical  construction  (C58) .   Another  way  of  looking  at  the 
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conjugate  point  condition  is  that   P  ,P  e.   UP   and   P   ^  P   yield  the  same 
point  on  an  extremal  trajectory,  i.e.,  adjacent  extremals  intersect.   By 
consideration  of  the  geometric  interpretation  of  (C58)  it  is  clear  that  for 
P   and  P„   to  yield  the  same  point  on  an  extremal: 

(1)  they  must  lie  on  the  same  circle  with  center  x  =  R,  y  =  0  , 

(2)  it  must  be  possible  to  obtain  P   from  P„   by  an  appropriate 
rotation  about   x  =  R,  y  =  0  given  by 

wi  wi  \     . 


\        /  1  1  \    • 

(I   -   x1w0)sin   s..    -  R\        /cos  —   (t„-t.)      sin  —    (t0-t1)\    (£-t0w   )sin   s„    -   R 
12  1  \  R         Z      1  R  2      1  22  Z  \ 


wi  ,  "i 


\    (l-T   w   )cos   s-j^         /       \^-sin  —   (T2*"Ti^      cos  ~^T   ^T2_TlV     '     (^-t2w2^cos   s2 

(C59) 
where   T*  -  t.  >  0  causes  a  rotation  in  the  negative  sense.   From  (C59)  it 

is  clearly  sufficient  that   Ti  =  To  =  ^^2    '      Also  if   Tl  =  T2  '  then  it:  is 
necessary  that   t   =  £/w   .   This  is  easily  seen  by  considering  (C59)  for 
■x      =   t ~    .      It  reduces  to 


' 


(2,  -  i1w?)sin  s.  -  R  =  (i   -  t.w  )sin  s   -  R  , 


and 


(I    -    T  W  )C0S  S-  =  tl    -    T2W  )C0S  S2 

Hence  if   s.  ^  s?    ,  we  must  have   x,w  =  t-w  =  I    .   Since  this  is  also 
sufficient,  it  suffices  to  consider   x  e  {x|£  -  tw  >  0}  .   Although  in 
further  research  we  have  not  been  able  to  prove  or  disprove  intersection  of 
adjacent  extremals  for   t  <_  £/w_  ,  we  have  shown  the  following: 

(a)   for  P   and  P9  e  UP   to  yield  the  same  point  on  an  extremal 
for   T-.  =  £/w   ,  it  is  necessary  that  T^  =  T2  ' 
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and   (b)   for   P..   and   P~  e  UP   and  such  that   0  <_  s   <  s   <^  S   to  yield 
the  same  point  on  an  extremal,  we  must  have  t   >  t_  . 

To  prove  (a)  we  proceed  as  follows.   For  P   and   P   to  lie  on  the 

same  circle,  by  (C59)  we  must  have 

2  2 

[(£  -  t1w2)sin  s-  -  R]   +  [U  -  t  w  )cos  s  ] 

2  2 

[  U  -  t„w  )sin  s2  -  R]  +   [Of   -  t.wJcos  s-]     , 

which  yields  the  following  quadratic  equation  for  w.  : 

2  2 
w  i        -   2t  {w  -  Rw2  sin  s2)  -  [2R£(sin  s2  -  sin  s  ) 

2   2 
-  2x  {w  -  Rw  sin  s  }   +  w   x   ]  =  0  , 

whose  solution  is  given  by 

w„t   =  (1  -  R  sin  s  )±/(w  t   -  1  +  Rsin  s„)   +  2R(sin  s„  -  sin  s  ) (l   -   t  w  ). 

Noting  that  when  s   =  s„  we  must  have   t   =  t~  ,  we  see  that  the  minus 
sign  is  extraneous  and  hence 

w?T9  =  (1  -  Rsin  s„)  +  /(w_t.  -  1  +  Rsin  s_)   +  2R(sin  s„  -  sin  s  )  (SL  -   t ,w  ) 

(C60) 
Assertion  (a)  follows  from  letting   t   =  l/w„    in  (C60) 

To  prove  (b)  for  I   -   tw.  >  0  ,  we  note  that  for   0  <_  s„  <  s,  £  tt/2 
we  have   sin  s„  <  sin  s..  .   Hence  using  (C60)  we  see  that 

w9t«  -  1  +  R  sin  s„  <  w_T   -  R  +  R  sin  s~ 


T-  >  T_  (C61) 

or  12 
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As  stated  above,  we  suspect  that  adjacent  extremals  do  not  intersect 
for   t  <  2,/w   ,  but  we  have  not  been  able  to  prove  this.   We  have  shown  that 
all  extremals  which  terminate  on  terminal  surface  for   0  <  s  <_  S  pass 
through  the  same  point  at   t  =  £/w_  .   Hence,  all  such  trajectories  (except 
the  barrier   s  =  S  )  terminate  at  this  point . 

d .   Geometry  of  the  Solution. 

In  this  section  we  summarize  the  results  of  analysis  by  drawing  a 
picture  of  the  optimal  trajectories  with  the  control,  $    ,  being  indicated 
in  various  regions.   The  singular  subarc ,   x  =  0  ,  and  part  of  the  paths 
leading  to  it  (we  have  not  done  this  analysis)  are  also  given.   As  noted  above 
the  entire  state  space  may  or  may  not  be  the  capture  region.   From  (C56) , 
we  let 


£*  =  R{/l  -  (w2/w1)2  +  (w2/w1)sin~1(w2/w1)  -  1}  .  (C62) 

Then  there  are  two  cases  to  consider 

(1)  I  <_  I*     barrier  meets  negative  y-axis 

(2)  I   >    I*     entire  state  space  is  capture  zone. 

In  case  (1)  the  Pursuer  can  only  achieve  capture  for  a  small  portion 
of  the  state  space  if  the  Evader  plays  properly.  This  is  shown  in  Figure 
C2,  in  which  only  the  right  barrier  is  shown. 

In  case  (2)  the  entire  state  space  is  the  capture  zone.   Optimal  tra- 
jectories for  this  case  are  shown  in  Figure  3.   There  are  two  further  cases. 
For   £/w  <  2R(tt  -  S)/w   ,  paths  terminating  on  UP   for   0  <  s  <_  S  all  con- 
verge to  point  A  at   t  =  £/w~.   This  is  the  case  shown  in  Figure  3.   When 
&/w9  >  2R(tt  -  S)/w,,  such  trajectories  meet  past  point  B  at  the  end  of  the 
barrier,  and  there   is  a  transition  surface  between  this  point  and  B.   As 
w?  ■>  0  ,  the  trajectories  approach  those  shown  in  Figure  B2. 
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Barrier 


Figure  C3.   Part  of  Optimal  Trajectories  for  Homicidal  Chauffeur  Game. 
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APPENDIX  D.   SURVEILLANCE-EVASION  GAME. 

In  this  appendix  we  derive  parts  of  the  solution  to  the  surveil- 
lance-evasion game  given  by  equation  (1)  of  the  main  text.   First,  we 
present  details  of  its  solution.   Next,  we  discuss  our  new  geometrical 
construction  for  optimal  trajectories  in  this  problem.   Finally,  we 
discuss  aspects  of  the  geometry  of  the  solution  and  summarize  analysis 
results  not  given  in  the  main  text.   A  more  detailed  discussion  of  many 
solution  steps  is  to  be  found  in  Appendix  B  (or  Appendix  C  if  they 
occur  there) . 

a.   Development  of  Solution. 
Hamiltonian,   H(t,x,p;4>,iJ;) 

The  Hamiltonian  for  the  problem  given  by  equation  (1)  of  the 
main  text  is  given  by 

Sl  Sl 

H(t,x,p;<}>,i}0  =  1  +  p^-  ^~  y<J>  +  s2  sin  i\i)   +  P2(^-  x4>     "  si  +  s2  cos  ^  '    (D1) 

r)  T  3J 

where   p. (t)  =  —  (t)   and  p0(t)  =  —  (t)   are  the  dual  variables  and 
1      3x  Z  dy 


9y 

T 


* 
J   =  max  min 

s,  A   s0,i|j 


'l'T  °2'r0       2,Y      l,r0 
from 


dt  =  min  max   dt.   We  determine  extremal  strategies 
so>^  si  '*n 


max  min  H(t,x,p;c(>,40 
s1^      s    ,\\> 


\A(-v,V  +   P„x)-pJ},  (D2) 

s. 


max  tSj^t-^C-p^  +  p2x)-p2]}, 


and 


min  (s?(p,  sin  i^+  p   cos  \p)} 


(D3) 


s2,i^ 
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It  is  clear  from  (D2)  (since   s  (t)  ^  0)   that   <J>   is  given  by 


(j>(t)  =  sgn  A(t)  , 


where 


A(t)  -  -pxy  +  p2x 


(D4) 


The  determination  of   s    is  more  complicated,  and  we  have  to  use 
results  to  be  established  later  in  this  appendix.   We  parametrically 
represent  the  terminal  surface  by   u   as  shown  in  Figure  Dl  below. 


Figure  Dl.   Terminal  Surface  for  Surveillance-evasion  Game. 
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Later  we  will  show  that  (where   t  =  T  -  t) 


R  Sl 

A(t)  =  -  {cos  u  -  cos(u  +  —  4>t)},  (D5) 

(p  K 


and 

Sl 


P2(t)  =  -cos(u  +  —  <j>T).  (D6) 

Substituting  the  above  into  (D2),  we  obtain 

max   {s1[^  A(t)  -  p„(t) ]} 
s,  ,<f> 

=  max{s1  cos  u},  (D7) 

Sl 
and  thus  we  see  that  (where   u  refers  to  termination  conditions) 

_  7T  TT 

f  Wl   f°r   "  2  *   U  <  2 


sx(t)  =1 


l-  0   for    |  <  u  <  |  Ti  (D8) 


Next,  we  consider  (D3).   To  minimize  with  respect  to  ty      it 
suffices  to  consider 


f(ip)  =  p1  sin  i>   +  p„  cos  v/j 


A  necessary  condition  is  that 


and  hence 


—  =  0  =  p   cos  ty   -   p0  sin  ip, 
dip        ±  / 


tan  <\>   =  Pi/P2* 


Thus,  to  minimize   f(ip)   we  must  have 


~pl  "P2 

sin  ij;  =        -    ,    cos  \\i   =    -  ,      (D9) 
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with  the  minimum  value  being 

min  f  0JO  =  -/p?  +  vi   <    °  > 
which  implies  that  (D3)  is  given  by 

lin   {s  f(i|i)}  =  min   {-s  /pjr  +  p^}, 
and,  hence,  we  see  that 


mj 

s2,^  s2 


s2(t)  =  w2.  (D10) 


Boundary  Conditions  for  Dual  Variables 

We  must  have  p(t  =  T)  =  (p  (T)  p?(T))   parallel  to  the  normal, 

n,   pointing  into  the  state  space  to  the  terminal  surface  (see  Figure 

->■-»■-»■ 

Dl).   Now,  since   n  =  -sin  u  i  -  cos  u  j,   we  have 


p  (t  =  T)  -  -sin  u,   p  (t  =  T)  =  -cos  u.  (Dll) 


Useable  Part  of  Terminal  Surface 


We  have  that 


n  =  -sm  u  i  -  cos  u  j 


-> 


■t  si  ->    si 

X  =    (-—  y<|>  +  s2   sin   uj ) ±  +    (—  x<J>  -  S-   +  s2   cos   iJOj    , 


where  the  terminal  surface  is  parametrically  represented  by   u  as 
shown  in  Figure  Dl  above.   Hence,  we  have 

x(t  =  T)  =  d  sin  u  and   y(t  =  T)  =  d  cos  u. 
Hence , 
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-   V  Sl 

n  •  X(t  =  T)  =  (-  —  <j)d  cos  u  +  s   sin  i|>)  (-sin  u) 

sl 

+  (^~  <t>d  sin  u  -  s  +  s   cos  \\>)  (-cos  u)  , 

and  when  we  recall  (see  (D9))  that 


-PiCt-T) 

sin  i^(t  =  T)  = =  sin  u  , 


-P2(t=T) 
cos  ijj(t  =  T)  =      ■  ■  ■-  =  cos  u  , 


we  obtain  from  n  •  X(t  =  T)  £  0   that 


n  •  X(t  =  T)  =  -s   +  s   cos  u  <;  0.  (D12) 

TT  3 

Recalling  (D8)  and  (D10)  ,  we  see  that  for  —  <  u  <  —  tt   (D12)  is 

TT  TT 

identically  satisfied.   For   -  —  <  u  <  —  ,   we  have 

-w  +  w  cos  u  £  0, 

where  we  also  assume  w   <  w    (otherwise  the  Evader  can  always  escape 
merely  by  "outrunning"  his  pursuer).   Hence,  the  useable  part  of  the 
terminal  surface  is  given  by 


UP  =  {u|u  £  u  <.   2tt  -  U  where   cos  U  =  w  /w  ,    0  <.   U  <:  tt  /  2 } .     (D13) 


We  note  that  this  is  the  complement  of  the   UP   in  the  homicidal 
chauffer  game  (see  (C7)  in  Appendix  C) . 
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The  Adjoint  Equations  and  Their  Solution 
We  have  that 


dpx 

dt 

SH  _ 
8x 

*** 

dp2 

dt 

3H 
9y 

&1  A 

Pl  T  ♦• 

To  integrate  backwards  from  time  of  escape,   T,   we  introduce  the 
"backwards  time"  variable   x   defined  by   t  =  T  -  t   and  obtain 


dpl      Sl 

dr~  =p2f*     P1(t  =  0)  =  ~sin  U' 


dp2       s1 

-fa-  -  -px  y~  <f>        p2(t   "  o)  ■   cos  u- 


We  combine  these  equations  to  obtain  a  second  order  differential 
equation  for  p   as  follows 

d2pl    dP2  "I  „       A  Ax2 


or 


d2pl    ,81 


71 


+    (—  <}.)  Pl  =  0. 


dTz      vR   t,  rj 


The  solution  to  the  above  equation  is  given  by 


Sl  Sl 

p..  (x)  =  A  cos  —  tj>x  +  B  sin  —  cj)T 
i  R  R 
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where  the  constants,   A  and   B,   are  determined  by 

p.  (t  =  0)  =  A  =  -sin  u 

dp  s       s  s 

—  (T  =  0)  =  —  <(,B  =  —  4>p2(T  =  0)=-^cosu 

or  B  =  -cos  u. 

Hence,  after  some  manipulation,  we  obtain 

Sl 
P1(x)  =  -sin(u  +  —  (J,t).  (D14) 

Similarly,  it  may  be  shown  that 

Sl 
p  (t)  =  -cos(u  +  —  <|)t).  (D15) 

Taking  note  of  equation  (D9),  we  also  obtain 

Sl  Sl 

sin  t/j   =   sin(u  +  — -  <j>x)      and      cos   ty   =   cos  (u  +  — -  <)>t).  (D16) 

R  R 

Taking  account  of  (D8),  we  obtain  from  the  above  (where   u   refers  to 

angle  at  which  escape  occurs,  see  Figure  Dl) 

7T         3  W2 

for  U  ^  u  <  —  and  —  n  <  u  ^  2 it  -  U   where   cos  U  =  — 
2       2  w 

and   0  £  U  £  — 

Wl  Wl 

P1(t)  =  -sin(u  +  —  <pj)      ,   p2(t)  =  -cos(u  +  —  <|>t)  ,     (D17) 

and 

Wl  Wl 

sin   \l>   =   sin(u  +  -—  <j>t)       ,      cos   ty   =   cos  (u  +  —  <J>t)    ,  (18) 

R  K 
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,  ,.      TT  3 

and  for  -r  <   u  <  —  it 

P1(t)  =  -sin  u   ,   p2(t)  =  -cos  u,  (D19) 

and 

sin  ty  -   sin  u    ,   cos  ij;  =  cos  u.  (D20) 


Solution  to  the  State  Equations 

In  the  "backwards  time"  we  have 

j  S  -i  s  - 

dr   =  R~~  *y   "   S2   sln^u  +  r~*^  X^T   =   °)    =   d   sin  u> 

d  Sl  Sl 

-j~-  =   -  — -  <j)x  +  sn    -   s     cos  (u  +  —  4>x)  y(r   =  0)   =   d   cos   u. 

dT  R  12  R 

The  above  equations  have   the   same   form  as   the   state   equations   for   the 

homicidal   chauffer  problem    (see  Appendix  C).      Hence,    their  solution  is 

given  by 

s  s 

x(t)   -    (d  -  xs2)sin(u  +^*t)+  ^(1  -  cos  ^  <f>x),  (D21) 

and 

s  s 

y(x)   =    (d  -  xs2)cos(u  +  -^-  <}>t)   +  ^  sin  ^  <f>T.  (D22) 

Taking  account  of    (D8) ,   we  obtain  from  the  above    (where     u     refers    to 

angle  at  which   escape  occurs,    see   Figure  Dl) 

it  3  W2 

for    U    £  u   <  —     and     —  tt   <   u  £  2tt   -  U     where     cos  U  =  — 
2  2  w 

and     0  ilisr 

w  w 

x(t)   =    (d  -   Tw2)sin(u  +  ^  <J>x)   +  ^(1  -   cos  ~  <J>t),  (D23) 


and 


y(t)   =    (d   -   xw2)cos(u  +  —  (fix)   +  —  sin  —  <|>t,  (D2A) 
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,  ,    tt       3 
and  for  —  <  u  <  —  tt 

x(t)  =  (d  -  tw  )sin  u   ,   y(x)  =  (d  -  tw  )cos  u.       (D25) 

We  note  that  equations  (D23)  and  (D24)  (and  also  (D17)  and  (D18)) 
hold  from  the  terminal  surface  (t  =  0)   until  a  transition  surface  is 
reached  or  the  trajectory  terminates. 

Equation  of  the  Barrier 

From  the  symmetry  of  problem,  we  consider  the  right  barrier, 

TT 

which  terminates  on  the  terminal  surface  with   u  =  W,   where   0  £.   U  <  — 
and 

cos  U  =  w2/w  .  (D26) 

For   u  =U   and   0  £  t  £  t  ,   we  have   <f(i)  =  1   (we  show  this  in  the 


next  section)  and  the  barrier  is  given  by 


for  0  £  t  £  t 

w  w 

x(t)  =  (d  -  xw2)sin(U  +  -i  T)  +  R(i  _  Cos  —  t),       (D27) 

and 

w  w 

y(x)  =  (d  -  tw  )cos(U  +  -i  t)  +  R  sin  —  t.  (D28) 


R   '    R 


We  now  consider 


w  w 

R  cos  -i  t  =  R  cos  (—  t  +  U  -  U) 

w  w 

=  R  cos  (-^   t  +  U)cos  U  +  R  sin  (— t  +  U)  sin  U 

R  ' 


Using  the  above  and  (D26) ,  we  obtain 
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Wl       W2     Wl  R   ,  Wl 

R  cos  -  i  =  R  —  cos  (—  t  +  U )  +  —  /  w'i   -  wi   sin  (~   x  +  U  )  , 
R        w      R  w      1     2      R 

which  we  combine  with  (D27)  to  obtain 

W?     W1  r   w9  wi       wi 

x(t)-R  =  -R  —  003(^1+  U)+{[d  -  —  /  wf  -  w^]-(R  -^)-±  T}sin(-iT+  U)  ,   (D29) 

W1       K  Wn      L  L  W.  K  K 

Similarly 


w      w 
y(x)  =  (R  — )sin(ri  x  +  u)+{ 

W.       K 


d  -  ^-  /  w*  -  W^ 
w     1     2 


w  w       w 

(R  -)A}cos(-ii+  U).   (D30) 

W1  K         K 


Recalling  (C30)  and  (C31) ,  we  see  that  the  above  are  the  equations  of 

an  involute  to  a  circle  of  radius   R  w„/wn   and  with  center   x  =  R,y  =  0. 

We  note  that  the  involute  winds  in  for 


d  -  R  A   -(w^v^)2  -  w2t  :> 


(D31) 


Transition  Surfaces 


We  recall  that 


<|>(t)  -  sgn  A(t)  , 


(D4) 


where  A(t)  =  -p,y  +  p9x 
Recalling  that 


x(t  =  0)  =  d  sin  u   ,   y(x  =  0)  =  d  cos  u, 


and 


p1 (x  =  0)  =  -sin  u   ,   p9(r   =  0)  =  -cos  u, 


we  see  that 


A(t  =  0)  =  0. 


(D32) 


Also, 


dA 
dr 


dp. 
dT 


4z  + 


dp, 


dx 


p,    •  f  ■    —  x  +  p_  —  > 
1  dx    dx       2  dx 
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into  which  we  substitute  the  state  and  adjoint  equations  to  obtain 

dA  Sl 

-pp  °  "P1S1  =  s1   sin(u  +  —  <j)x)   with  A(t  =  0)  =  0. 

Integration  of  the  above  yields 

R  Sl 

A(t)  =  -   {cos  u  -  cos(u  +  — -  4>t  )}•  (D33) 

<p  R 

Recalling  (D8)  and  noting  that  for  the  escape  paths  terminating  with 

TT  3 

■r  <  u  <  —  tt   the  Pursuer  uses   s.  (t)  =  0,   we  see  that  in  this  range 
there  is  no  transition  surface  since  A(t)  5  0   for  all  time.   Hence, 
we  consider  only  paths  for  which  there  is  a  transition  surface,  i.e., 

TT  3 

U^  u   <  —     and     —  tt    <    u  <:   2tt    -   U.       in   this    range   of      u      by    (D8)    we   have 
s      =  w.       and      (D33)    becomes 

R  Wl 

A(t)    =  —  {cos   u   -   cos(u  +  — -  <J>t)}.  (D34) 

<J>  K 

Thus,    for      0  £   t   £   t         (where   we    determine      x        below), 


<J>(t)    = 


+1      for    U     ^   u   <   — 


-1      for     |  tt   <   u  <;  2tt   -    U,  (D35) 


where  U   is  given  by  (D26).   By  the  symmetry  of  the  problem  we  concen- 


TT 


trate  on  the  right  half-plane.   Consequently,  for  ^  £  u  <  — ,   we 
determine   T    as  follows:   it  is  the  first  time  that   A(x)  =  0   after 
t  =  0.   This  happens  when 


Wl 

COS  U  =  COS (u  +  —  T^) 


which   is   precisely  when 


Wl 

2tt    -    u   =    u   +  —   T 


100 


7T 

Hence,  for  U  ^  u  <  — ,   we  have 

2R 
t.  '*  —  (tt  -  u).  (D36) 

i.    w1 

It  appears  as  though  all  trajectories  except  the  barrier  (u  =  U) 
terminate  before  this  time,  however,  because  of  intersection  of 
adjacent  extremals.   Hence,  equation  (D36)  becomes 

T   -  f±    (TT  -  U),  (D37) 

1   w± 

where  U   is  given  by  (D26).   For   x  >   t  ,   we  show  later  that  A(t)  <  0 
and  hence   <)>(t)  -  -1-   We  finally  note  that  (D37)  holds  only  if  the 
barrier  doesn't  terminate  before  this  time  is  reached. 


Termination  of  the  Barrier 

A  fundamental  difference  between  the  homicidal  chauffer  game 
and  the  surveillance-evasion  game  is  that  in  the  latter  the  barrier  may 
terminate  abruptly  before  the  transition  given  by  (D37)  occurs.   We 
recall  (see  Appendix  C)  that  in  the  homicidal  chauffer  game  the  barrier 
terminated  when   <J>(t)  changed  from  +1   to   -1.   In  the  surveillance- 
evasion  game  we  shall  see  that  for 

d  <  R  {/l  -  (w2/w1)2  +  2(w2/w1Xtt  -U)  ,  (D38) 

the  barrier  terminates  at 

t   =  (d  -  R  sin  U)/w  .  (D39) 

We  have  investigated  many  aspects  of  this  phenomena  and  will  detail  our 
findings  here. 
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What  led  us  to  our  findings  was  a  statement  by  Dobbie  [14]  which 
we  found  to  be  incorrect.   Let  us  consider  Figure  5  of  the  main  text. 
Dobbie  ([14]  p.  176)  states  that  the  (right)  barrier  can  be  tangent  to 
the  line  through   0   and   D'   (if   d   is  small  enough).   This  was  found 
to  be  incorrect.   Let   v   be  a  unit  vector  parallel  to  line  connecting 
0   and   D'   and   v   tobeaunit  vector  orthogonal  to   v  .   Thus 


v  =  sin  U  i  -  cos  U  j , 


v  =  cos  U  i  +  sin  U  j . 


(D40) 


We  show  below  that  there  is  a  transition  from   <J)  =  1   to  <p   =  -1,   i.e,. 
A(t)   changes  from  positive  to  negative,  at   T   =  2R(tt  -  U)/w  .   Now 
we  will  show  that  we  always  have 


dX 


,dx. 


T  =  T. 


vx  =  0, 


(D41) 


i.e.,  the  tangent  to  the  barrier  is  orthogonal  to  the  line  through  0 

0   only  for   d  -  x  w  =  R  sin  U, 


and  D'   at   x  =  x..  .   Also, 

dX 


dX 


ldTJ 


T  =  T. 


2 


12 


when  we  have  that  -r-  =  0.   Hence  the  barrier  can  never  be  tangent  to 

dx 

this  line.   We  should  note  that  for   t  =  t1   our  new  geometric  construc- 
tion (discussed  below)  shows  that  the  point  (x (x  ),y(x-))   of  the 
barrier  always  lies  on  the  line  through   0   and   D'   (if  we  ignore 
termination) . 

To  prove  (D41)  we  consider  (1)  of  the  main  text  at   x   =  2R/w  (tt  -  U) 


dx 


and 


dx 


dy_ 
dx 


v . 


x=x. 


x=x. 


—  (d  -  x  w  )cos  U  -  w   sin  2  U  +  w_sin  U , 
R        1  2  1  2. 


Wl 

—  (d  -  x  w  )sin  U  +  w  cos  2  U  -  w  cos  U, 

R         i  Z.  J-  *- 


(D42) 
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where  we  have  also  used  (D27)  and  (D28) .   Now 


'd£ 


dx 


T  =  T. 


'  Vl  = 


dx 
dr 


sin  U  - 


T=T. 


cos  U  , 


T  =  T. 


which  use  of  (D42)  and  some  manipulation  shows  to  be  equal  to  w, 
-w   cos  U  =  0,   and  hence  (D41)  is  proved. 

The  second  statement  we  prove  similarly 


or 


dX 


dx 


'  T=T. 


dx 

dTj 


COS     + 


T=T. 


d^_ 

dr 


sin 


U 


T  =  T. 


dX 

idx; 


T  =  T. 


W, 


v  =  —  {d  -  t.w  -  R  sin  U}, 


(D43) 


where  we  have  used  (D42)  and  simplified.   Thus, 


di 


•  v  =  0 

vdTJT=T,      2 


for   d  -  x,w  =  R  sin  U  .   If  we  substitute  this  latter  condition  into 
(D42)  we  find  that 


dx 

Idx 


T=T. 


=  sin  U  {-w   cos  U  +  w  }  =  0, 


and 


dX 


=  cos  U  (w.  cos  U  -  w„)  =  0. 
dTJT=T  1  2 

1 


Let  us  now  discuss  how  when  (D38)  holds  the  barrier  has  a  cusp 
at   f  =  f   given  by  (D39).   We  obtained  several  clues  by  considering 
our  new  geometric  interpretation  of  the  solution  to  the  state  equations, 
which  we  now  discuss.   The  equations  of  the  barrier  (D27)  and  (D28)  may 
be  written  as 
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Wl  wi 

x(t)-  R  =  (d  -  tw  )cos  U  sin  —  x  +  { (D  -  tw  )sin  U  -  R}cos  —■   t, 


and 


v  w 

v(t)  =  -{(d  -  tw  )sin  U  -  R}sin  —  t  +  (d  -  tw  )cos  U  cos  ==  t, 


which  we  may  write  as 


for  0  £  t  £  t 


w. 


x(t)-  R 

yd) 


w. 


cos  —  t     sin  —  T 

K  R 


w  w 

-sin  —  t    cos  —  t 

K  R 


(d  -  tw  )sin  U  -  R 


(d  -  tw  )cos  U 


(D44) 


Let  us  consider  Figure  5  of  the  main  text.   To  determine  a  point  on  the 
right  barrier  at  time   t  £  2R/w  (tt  -U),   we  start  at  point  A  at  the 
boundary  of  the  detection  region  and  move  to  a  point  which  is  a  distance 

tw   away  from  A  along  the  straight  line   OA.   We  now  rotate  this 

Wl 
point  through  an  angle  —  t   in  the  negative  (clockwise)  sense  about 

R 

the  point   x  =  R,y  =  0. 

Some  further  remarks  seem  appropriate.   When   t  =  T.   we  rotate 
through  an  angle   2tt  -  2U   in  the  negative  sense  or  an  angle  of   2U 
in  the  positive  sense.   Hence,  any  point  on  the  straight  line  through 
0  and  A   is  carried  to  the  line  through   0   and   D'.   Hence,  our  remark 
about  a  point  of  the  barrier  being  on  OD '   at   t  =  t. .   We  also  see 
the  suggestion  of  anomalous  behavior  if  the  point  moving  along   OA 
passes  the  point  of  tangency  to  the  circle  of  radius   Rw  /w   and  center 
x  =  R^y  =  o.   If  this  occurs  the  geometry  of  the  situation  tells  us  that 
the  point  of  the  barrier  for  t  =  t.   lies  on  OD'   above   D'   toward 
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the  boundary  of  the  detection  region.   As  the  point  moves  along   OA 
toward   0  before  rotation  we  see  that 

(a)  when  it  is  above  the  point  of  tangency  to  the  circle, 
the  resultant  point  (after  rotation)  is  moving  toward 
this  circle, 

(b)  when  it  is  at  the  point  of  tangency,  the  resultant  point 
is  on  the  circle  of  radius   Rw  /w  , 

and    (c)   when  it  moves  past  the  point  of  tangency  for   r  <  T, , 

the  resultant  point  moves  away  from  the  circle. 
Again  we  note  that  this  is  because  the  construction  rotates  the  point 
about   x  =  R,y  =  0. 

Algebraically,  we  can  see  the  above  approach  and  recession  by 
considering  (D44)  from  which  we  can  obtain  the  square  of  the  distance, 
denoted  by  D(t),   of  a  point  on  the  barrier  at   t   from  x  =  R,y  =  0 


D(t)  =  (x(t)-  R)2  +  y2(x)  =  (d  -  xw2  -  R)2  +  2R(1  -  sin  U)  (d  -  xw2). 
Hence,  for  the  minimum  of  D(t),   we  must  necessarily  have  — j =  0, 


which  yields 


t2  =  (d  -  R  sin  U)/w2.  (D45) 


This  value  of   t   does  indeed  yield  a  minimum  since   ,  ?  =  2(w^)2  >  0. 

dx       I 

Hence,  we  suggest  that  the  barrier  has  a  cusp  at  such  a  point, 
if  it  occurs.   Let  us  further  note  that  such  behavior  can  occur  on  any 
optimal  trajectory  terminating  on  the  boundary  of  the  detection  region 
for  U  <  u  <  — .   Thus,  it  appears  that  a  trajectory  terminating  at  a 
point   P   on  the  boundary  (if   d   is  right)  similarly  approaches  and 
receeds  from  the  circle  to  which  OP   is  tangent.   We  have  not  had  time 
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to  explore  this  further.   Let  us  note  that  this  doesn't  occur  for 
u  =  —.      We  note  that  for  x    to  occur  before   x   we  must  have 

d  -  R  sin  U  <  2R  w  (it  -  U)/w 


or 


d  <  R{sin  U  +  2  w2(tt  -  U)/w;L 


which  by  (D26)  is  precisely  the  condition  given  by  (D38) . 

Now  that  we  are  suspicious  that  if  (D38)  holds,  the  barrier 
has  a  cusp  in  it  if  it  doesn't  terminate  at   x„,   let  us  prove  the 
existence  of  the  cusp  algebraically  before  we  plot  the  curve  of  the 
barrier.   Differentiating  (D23)  ,  we  obtain 


WW  WW 

~  =   -w2   sin(U+  ^  T)+  ~  {(d   -  xw2)cos(U+  ^  x)  +  R  sin  ~-  x}.  (D46) 

Noting    that 

Wl  Wl 

-w  sin(U  +  —  x)+  w     sin(—  x   +   U  -  U)    = 
z  K  IK 


-w-    sin 


w  w  w 

(U  +  -—  t)+  w.    sin(U  +  — -  x)cos  U    -  w     cos(—  x   +  U)sin  U 
R  1  R  IK. 


Wl 
=  -w     cos(—  x   +  U)    sin   U, 

1  K 


where  we  have  used  (D26) ,  we  see  that  (D46)  is  equal  to 


^   =  -wn  cos(U+  -i  t)  {sinU-(d  -  x  w.)/R}.        (D^7) 
dx     1         R  ^ 
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Similarly 

d  wl 

-^  =  W;L  sin(U  +  —  x){sin  U  -  (d  -  xw2)/R},         (D48) 

and  hence 

d  W1 

^  =  -tan(U+  ^  t).  (D49) 

Thus,  we  see  that  -7^-  is  continuous  at   t  =  x„  =  (d  -  R  sin^)/w„. 

dx  11 

dx     ,   dy  , 

but  both  —  and  -f-   =  0   at   x  =  x„   and  change  sign  as   t   passes 
dx       dx  I 

through   x~.   Hence,  the  barrier  curve   y  =  y(x)  has  a  cusp  at   x  =  x„. 
Thus,  the  barrier  must  terminate  (see  Appendix  B  for  discussion)  and 
(D38)  and  (D39)  have  been  proved. 

Another  way  to  see  this  is  to  consider  (D29)  and  (D30) .   From  the 
discussion  of  the  involute  in  Appendix  C,  we  see  that  the  involute 
winds  in  for 


w   w 


d  -  R/l  -(w.AO*  -  (R— )(-±  x)  ;>  0, 
2   1        w..   R 


Therefore  if 


T-  =  —  2(tt  -  U), 
1    w 


doesn't  occur  before  the  involute  touches  the  circle  with  radius 

R  w_/w  ,   the  involute  starts  to  unwind.   But  for  this  to  hold,  (D38) 

follows  immediately.   We  note  that  the  involute  unwinds  for 

w   w 
{[d  -  R/l  -(w2/W;Ln  -  (R  ^)(~-   t)}  <  0. 

Figure  D2  shows  the  cusp  which  would  occur  if  barrier  didn't 

terminate.   The  values  of  parameters  which  were  used  to  calculate  the 

curve  are  shown  on  the  figure. 
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• 


y  n 


w  =  20   R  =  20 
w  =  10   d  =  10  tt 


•< 


Figure  D2.   Occurrence  of  Cusp  if  Barrier  Does  Not  Terminate. 
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Continuation  of  Solution  Past  Transition  Point 

It  may  be  shown  that  when  barrier  does  not  terminate  (see  (D38)) 

other  extremals  intersect  before   x  =  x    and  hence  terminate,  so  we 

the 
consider  justi(right)  barrier.   From  (D17) ,  we  have 

p,  (t.)  =  sin  U    ,   P2(T-j)  =  ~cos  ^  • 


From  (D27)  and  (D28),  we  obtain 


x(x  )  =  (d  -  t  w  )(-sin  U)  +  R(l  -  cos  2  U)  , 

y(t1)  =  (d  -  x  w  )  cos  U  -  R  sin  2  u  .  (50) 

Later,  we  show  that   4>(T)  =  ~1   f°r   T  >  Ti*   ^e  assume  this  for  now. 
The  adjoint  equations  for   x  >  x    are 


dpl 

ch-  =  -p2w1/R     P]/1].)  =  sin  U  » 


dp2 

dx"  =  piwi^R      P2^T1^  =  ~C0S  U 


The  solution  to  the  above  equations  is  (for   x  >  t  ) 


Wl 
P1(t)  =  sin(U  +  —  (t  -  t  ))  =  -sin  \\) , 


w 


P2(t)  =  -cos(U  +  —  (t  -  t  ))  =  -cos  i>.  (51) 


The  state  equations  for   t  >  x    are 
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dx     Wl  Wl 

df=  "  R-  y  +W2  sin(^+r(x  -  Tl)), 


w 


dy_    1 


dx  Wl  -  W2  cos(U  +  ^(T  "  Tl})' 


with  initial  conditions  given  by  (D50).   A  rather  laborious  computation 
(we  omit  the  details)  yields  the  solution  to  the  above  as  (for   x  >  x  ) 


Wl 
x(x)  =  -(d  -  xw2)sin(U  +  —  (x  -  x^) 


w  w 

+  R{2  cos  ~^(x  -  xx)-  1  -  cos(2  U  +  —  (T  -  x^)}, 


and 


w . 


y(x)  =  (d  -  xw  )cos(U  +  ^-(t  -  t  )) 


w  w 

+  R{2  sin  ^(x  -  x1)-  sin(2  U  +  —  (x  -  x^)} 


(52) 


We  may  write  equations  (D52)  as  (for   x  >  x  ) 


x(x)+R" 


y(T), 


wi  wi 

cos  — (x-x  )  U  -sin  ^~(T_Ti) 


wl  Wl 

[sin  —  (t-t1)    cos  — (t-t1)J 


(d-xw  )  (-sin  U)+  2R  -  R  cos  2^ 


(d-xw  ) cos  U  -  R  sin  2U 


(D53) 


or  using  (D50) 
x(x)+R 


w. 


.  y(T), 


1/     n       t       '"*-t  ^ 

cos  —  (x-x1)    -sin  —  (x-x., 


Wl 
sin  —  (x-x^     cos— (t-t.,)J 


K 


x(T1)+  w2(x-x1)cos(|  -  U)  +  R 
t  y(x1)-w2(x-i1)sin(^  -  U) 


.   (D54) 
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The  most  important  aspects  of  the  geometric  interpretation  of 
(D54)  are  that  rotation  is  in  the  positive  (counter-clockwise)  sense 
about  the  point   x  =  -R,y  =  0   and  the  involute  unwinds.   That  (D52) 
does  indeed  trace  out  an  involute  may  be  seen  by  writing  it  in  the 
equivalent  form 


w  w 

x(t)   +  R  =  R  —  cos(U  +  ~(x   -   t    )) 
w..  R  1 


/  W        o  WW  w 

+    [{3R/1  -M)      -  d}   +    (R  -hit  -i)      sin(U  +  -i(T   -   t    )), 

W-  W,  i\  K.  _L 


w      2  wo  wi  wi 


y(x)    =   -    [{3R/l   -i-1)2-   d}   +    (R  -^)(x  -1)]cos(U  +  -± 


Lwn '  w  R 


w  w 

+  R  —  sin(U  +  -i(T    -   Tl)),  (D55) 


w2 
which   is   the   involute   to   a  circle   of   radius     R  — *-     and  with   center 

Wl 
x   =   -R,y   =   0. 


Shape  of  the  Surveillance  Region 

When  a  surveillance  region  does  exist,  it  may  take  on  one  of  two 
shapes  depending  on  whether  or  not  the  barrier  terminates  by  intersect- 
ing the  negative   y-axis .   These  two  possibilities  are  shown  in  Figure 
5  and  6  in  the  main  text  where  further  details  are  given.   We  develop 
here  the  condition  for  the  barrier  to  look  like  that  shown  in  Figure  6, 

d  :>  R{/1  -(w2/w1)2  +(w2/w1)(  |  tt  -  U)+  1},  (D56) 


Ill 


which  we  may  also  write  as 


-1 


d  ^  R{/1  -(w2/Wl)2  +(w2/Wl)sin  i(-w2/w  )+  1},        (D57) 


where 


7T  «s  sin   (-w  /w  )  ^  3/2  tt. 


We  consider  Figure  D3  (which  isn't  drawn  to  scale)  and  recall  the 
relationship  between  two  normals  to  the  involute  of  a  circle  given  by 
(C32)  (see  Appendix  C) .   In  Figure  D3,  we  have 

ll  +   l2   =  £3  '  (D58) 

i.e.,  the  difference  in  the  length  of  the  normals  is  equal  to  the  distance 
on  the  perimeter  of  the  circle  (evolute)  between  the  points  of  tangency. 
When  the  barrier  intersects  the  negative  y-axis  we  have 


R  <;  l    .  (D59) 


We  also  have 


£0  =  d  -  I,  or 

3        4 


I      =  d  -  R  sin  U,  (D60) 


and 


where 


I     =   R  w  /w  a  ,  (D61) 


2  (tt  -  U)  =  a  +  6  or   a  =  2tt  -  2U  -(tt/2  -  U)  , 

or 

a  =  3/2  tt  -  U.  (D62) 


112 


\ 


\ 


U  -* 


/ 


^ 


Figure  D3.   Determination  of  Tangency  Condition 
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Combining  (D58),  (D59) ,  (D60) ,  (D61),  and  (D62) ,  yields 

R  +  R(w2/w])(3/2  tt  -  U)  <:  d  -  R  sin  U  ,  (D63) 

which  by  use  of  (D26)  is  seen  to  be  (D56).   Noting  that 


we  see  that 


w  t 

^/=  (3/2, -U), 


Wl  l2 
sin(—  — )   =  -  cos  U  =  -  w  /w  , 


and  hence 


S, 2  =(Rw2/w1)sin   (-w2/w  )  , 


which  leads  to  (D57) 


Absence  of  Singular  Solution 

Unlike  the  problems  considered  in  Appendices  B  and  C,  there  does 
not  appear  to  be  a  singular  solution  to  this  problem. 

Another  Way  to  Determine  UP 

We  have  seen  (see  Appendix  B)  that  the  useable  part  of  the  terminal 

-> 
•"+■-*■ 

surface  may  be  determined  from  X  •  n  £  0,   where  n   is  a  unit  normal 

vector  to  the  terminal  surface  and  points  into  the  state  space.   Another 
criterion  used  by  Isaacs  ([30]  p.  239)  in  a  capture  game  is  that  the 
UP   is  determined  by  —  (r2)  £  0,   where   x2  +  y2  =  r2.   For  the  escape 
problem  at  hand,  this  condition  becomes 
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(r  r)    ^  0.  (D64) 

r=d 


[We  note  that  this  condition  is  examined  only  on  the  terminal  surface 
and  that  optimal  strategies  are  not  determined  by  the  rate  of  change 
of  range  as  one  might  infer  from  Dobbie  [14]  pp.  175-176  (see  [30] 
pp.  205-206).] 

Using  (1)  and  (D10) ,  we  obtain  from  (D64) 

w  (x  sin  \\>   +  y  cos  \\>)-   s  y  ^  0,  (D65) 

2  1 

which  becomes  for   x  =  0   and  using  (D20) 


dw  (sin2  u  +  cos2  u)  ^  s  d  cos  u, 


whence  letting   s   =  w 


w  -  w   cos  u  ^  0,  (D66) 


->- 

•    -> 
which  is  the  same  result  as  from  X  •  n  s;  0. 


b.   The  New  Geometrical  Construction  for  Optimal  Trajectories. 

We  consider  optimal  trajectories  in  the  right  half-plane  for 
0  £  x  £  x   =  2R/w  (tt  -  u)   and  U  ^  u  <  tt/2  ,   cos  ^  =  w  /w  .   Background 
material  is  to  be  found  in  Appendix  B.   For   0  £  x  ^  x1 ,   we  have  that 
<|>(t)  =  1,   and  by  (D23)  and  (D24)  we  have 


Wl  Wl 

x(x)  -  R  =  (d  -  xw  )cos  u  sin  —  x +  { (d  -  xw  )sin  u  -  R}cos  —  x, 

Z  K  Z  K 

and 

Wl  Wl 

y("r)  =  -  {(d  -  xw  )sin  u  -  R}sin  —  x  +  (d  -  xw  )cos  u  cos  ^—  x,     (D67) 
2.  K  Z  K 
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which  we  may  write  as 


for  U  ^  u  <  tt/2  where   cos  U  =  w  /w   and   0  £  x  £  x 


w. 


w. 


x(x)-  R 

yd) 


cos  —  x   sxn  —  x 
R  R 


w. 


w. 


-sin  —  x   cos  —  xj 


(d  -  xw  )sin  u  -  R 


(d  -  xw  )cos  u 


(D68) 


Equation  (D68)  appears  in  the  main  text  as  (3).   The  geometric  interpre- 
tation of  these  equations  follows  equation  (3)  in  the  main  text. 

Considering  the  geometric  interpretation  of  (D68) ,  we  see  that 
the  involute  winds  in,  i.e.,  radius  of  curvature  decreases  with 
increasing   x,   for   0  £  x  £  x  .   Further  geometric  investigations 
have  shown  that  neighboring  extremals  intersect  (see  Figure  3).   This 
means  that  on  an  extremal  there  is  a  point  conjugate  to   x  =  0   and 
that  the  trajectory  terminates  here  (except  for  the  barrier).   Time  has 
not  permitted  this  to  be  more  fully  investigated. 


c .   Geometry  of  the  Solution. 

This  topic  is  discussed  in  section  Hid  of  the  main  text 
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APPENDIX  E 

SOME  ALLOCATION  OF  SEARCH  EFFORT  RESULTS 
BY  THE  PONTRYAGIN  MAXIMUM  PRINCIPLE 

A  major  problem  in  Naval  warfare  is  the  search  for  targets  at  sea. 
Hence,  the  optimum  allocation  of  search  effort  is  of  interest  in  developing 
tactics.   We  show  in  this  appendix  that  some  well— known  results  in  the 
optimum  allocation  of  search  effort  may  be  more  easily  obtained  by  appli- 
cation of  the  Pontryagin  maximum  principle  [43]  and  the  "sharpness"  of 
the  results  extended  slightly.   We  begin  by  reviewing  briefly  the  litera- 
ture. 

Koopman  wrote  the  first  major  work  on  search  theory  [35] ,  which  was 
for  a  long  time  classified  (until  May  10,  1961).   It  still  remains  a 
major  work,  especially  remarkable  for  containing  many  of  the  concepts  for 
research  being  performed  20  years  later.   Som  examples;   formulation  of  non- 
linear programming  problem  for  searching  ocean  areas,  Bayesian  approach  to 
sequential  search.   Later,  Professor  Koopman  published  some  of  these  results 
in  the  open  literature  [36],  [37],  [38].   In  [38],  Koopman  solved  the  problem 
for  a  continuous,  one-dimensional  model  of  the  optimum  distribution  of  search 
density  when  the  conditional  probability  of  detecting  the  target  is  exponen- 
tial.  However,  it  should  be  noted  that  Chapter  3  of  [35]  contains  much 
material  not  in  [38].   For  example,  in  [35]  Koopman  first  solves  a  discrete 
problem  of  searching  effort,  discusses  the  physical  interpretation  of  various 
quantities  and  the  structure  of  the  solution,  and  then  extends  this  to  the 
case  where  targets  are  continuously  distributed  (two-dimensional) . 

Charnes  and  Cooper  [10]  extended  [38]  by  formulating  a  non-linear 
programming  model  for  searching  discrete  alternatives.   They  solved  this 
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model  by  applying  the  Kuhn-Tucker  conditions.   De  Guenin  [13]  extended 
Koopman's  results  [38]  by  considering  the  optimum  distribution  of  search 
density  for  a  general  conditional  probability  of  target  detection  (he 
assumed  this  to  be  a  concave  function  of  search  density). 

We  now  derive  the  solution  to  Koopman's  problem  [38]  and  de  Guenin' s 
[13]  by  the  maximum  principle.   The  reader  is  directed  to  these  papers  for 
model  formulation.   Even  though  Koopman's  problem  is  a  special  case  of 
de  Guenin's,  there  still  appears  to  be  methodological  value  in  considering 
it  first. 

a.   Koopman's  Problem. 

The  problem  studied  by  Koopman  may  be  stated  as  to  maximize  target 
detection  probability  when  there  is  a  restriction  on  the  total  amount  of 
search  effort  available.  Mathematically,  the  problem  may  be  stated  as 


maximize 
+ 


subject  to: 


p(x){l  -  e""^00}  dx   , 


(x)  dx  = 


and  <j>(x)  >_  0  »  (E1) 

where 

p(x)dx  =  Prob[ target  located  between  x  and  x  +  dx  ]  , 

1  _  e-(Hx)  _  prob[detect  target  with  effort  <j>  (x)  |  target  located  at  x  ]  , 

<J>(x)    is  search  density  and  defines  the  distribution  of  search  effort, 


and      $    is  total  search  effort 
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We  note  that 


p(x)  dx  =  1    and  p(x)  ±_   0 


We  now  consider  the  following  equivalent  optimal  control  problem: 


maximize 


p(x)U  -  e  *(x)}  dx  , 


subject  to:      dy/dx  =  cf>(x)  > 
where      <j>  (x)   >   0  , 


and 


y(x  =  — °°)  =  0   and   y(x  =  +°°)  = 


(E2) 


The  Hamiltonian   for   this  problem   is 

H(x,y,A,<j>)    =   p(x){l   -   e"*(x)}      +      X<|>  (x)    , 


where  A   is  the  dual  variable  corresponding  to  the  state  equation. 


(E3) 


.00 


Defining  J*  as  equal  to  max 


p(x){l  -  e     }dx  ,   we  see  that 


A  =  8J*/8y  <   0   , 


since  y(x)   is  cumulative  effort   (y(x)  = 


rx 


4>  (x)  dx)   and  by  expending 


more  effort  than  is  optimal  we  can  do  nothing  but  reduce  the  optimum  target 
detection  probability.   By  the  condition 


d_A 
dx 


3JH. 

3y 


=  0 


we  see  that   A (x)   is  a  constant. 

The  optimum  distribution  of  search  effort  is  found  by  maximizing  the 
Hamiltonian  with  respect  to  the  control  variable  <j>  ,  which  is  the  search 
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density.   To  facilitate  a  later  argument,  we  let  X   =   -u  where  u  >  0 
Then,  the  Hamiltonian  is  maximized  by 


3H/3<j)  =  p(x)e~^(x) 


V   =  0     for     <j>Cx)  >  0 


or 


3H/3cj)  <  0   for     <j)(x)  =  0 

Since   3  H/34>   =  -p(x)e      <  0   for  all  x  and  <j>(x)  <  «°  ,  we  see  that 
sufficient  conditions  for  a  global  maximum  are  satisfied,  i.e.,  H(4>)   is 
concave.   Thus,  we  see  that  the  optimum  distribution  of  search  effort  is 
determined  as  follows: 

(a)  for  p(x)  <  u,   <Kx)  =  0  ,  since  then  p(x)e  ^    <  u   or   3H/34>  <  0  , 

~4>  (x) 

(b)  for  p(x)    >_  y,      <|)(x)    =   £n(p(x)/u)    ,    since     p(x)e  =  u      yields 

3H/3(j)      =      0  (E4) 

We  determine  u   as  follows.   Define  0,  =      {x|p(x)  >_  \}}    .   Then  u      is  chosen 
so  that 


£n(p(x)/y)dx  = 


<J)(x)dx  =  $ 


When  the  appropriate  sufficient  conditions  (see  [9]  pp.  181-182)  are  checked, 
i.e.  strengthened  Weierstrass,  strengthened  Legendre,  and  Jacobi,  it  is  found 
that  (E4)  is  both  necessary  and  sufficient. 

b.   De  Guenin's  Problem. 

Here,  we  consider  a  more  general  conditional  probability  of  target 
detection.   The  problem  is 
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maximize 

4 


pCx)h(4>(x))dx   , 


subject  to: 


<j)(x)dx  =  $ 


and 


(x)  :■  0  , 


(E5) 


where  p(x)  ,   <j>(x),   and  $  are  defined  as  before  and 

h(4>(x))  =  Prob[detect  target  with  effort   <f>  (x)  |  target  is  at  x  ]  . 

De  Guenin  further  assumes  that   h((j))   is  concave  with  b,lGj>  ~  0)    0  , 
i.e.,  h'(<£)  =  dh/dtj)   is  marginal  return  of  conditional  detection  probability 
with  search  effort.   Diminishing  returns  may  be  stated  as  h"(cf>)  <  0  with 
dh/d<j)  (<}>  =  0)  >  0  and   dh/d(J>  (<j>  =  °°)  =  0  .    It  is  noted  that  the  condition 


h"(c)))  <  0  for  all  <f>   implies  that  the  inverse  of  hr  (<j>),  i.e.   h 
well-defined. 

Again,  we  consider  an  equivalent  optimal  control  problem: 


,-1 


,  is 


maximize 


p(x)h(4>(x))  dx 


subject  to 


dy/dx  =  <f>(x)  , 


where 
and 


>(x)  ^0 
y(x  =  -oo)  =  o   and   y(x  =  +00) 


(E6) 


The  Hamiltonian  for  this  problem  is 


H(x,y,A,cj>)  =  p(x)h(4.(x))  +  A<Kx) 


(E7) 
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where,  as  before,  the  dual  variable,   A  ,   will  turn  out  to  be  negative 

A  =  3J*/3y  <  0 


As  before 


dA/dx  -  -3H/9y  =  0  , 


so  the  dual  variable  is  a  constant,  which  we,  for  convenience,  set  equal  to 
-u  where  u  >  0  .   Maximization  of  the  Hamiltonian  with  respect  to  the 
control  variable  <J>   is  determined  by 

3H/34>  =  p(x)h'(4>)  -  u  =  0     for     <|>  (x)  >  0  , 
or 

9H/84)  <  0        for        <(>(x5  -  0  , 

which  is  sufficient  for  a  global  maximum,  since 

82H/3cj)2  =  p(x)h"(<J>)  <  0     for  all    x  and   <j)(x)  • 
We  determine  the  desired  search  density,   <J>  ,  as  follows: 

<   0  , 


3H 
(a)   for  p(x)h'(cj>  =  0)  <  u,   <J>(x)  =  0  ,  since  then  t-t 

3<p 


*=0 


(b)   for  p(x)hf(<j)  =  0)  >_y,   <()(x)  =  h'  1(u/p(x))  ,  since  h'  (<j>)  =  u/p(x) 

yields  |^  =  0  ,  (E8) 

where   u   is  determined  similarly  to  the  previous  case.   It  is  easily  shown 
that  (E8)  are  both  necessary  and  sufficient  for  the  optimum  distribution 
of  search  effort.   Thus,  we  have  shown  that  de  Guenin's  results  are  also 
sufficient  for  the  determination  of  optimum  search  effort. 


122 


c .   Extensions . 

Here,  we  mention  some  extensions  of  either  analysis  or  models  to  the 
above.   It  can  be  shown  that  the  above  results  may  be  developed  strictly 
within  the  framework  of  the  classical  calculus  of  variations  (and  in  a 
different  way  than  reported  in  the  literature)  but  this  analysis  has  not 
been  completely  documented  at  this  time.   Besides  the  trivial  extension  to 
h  =  h(x,(f)(x))  ,  already  noted  by  de  Guenin,  we  may  extend  the  maximum 
principle  approach  to  some  cases  where  h(<j))   is  not  concave. 
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